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Accountability 
and  Evaluation 


what  Does  The 
Federal  Partner  Want? 


Believe  it  or  not,  the  federal  partner 
is  interested  in  the  same  kinds  of 
accountability  and  evaluation 
information  that  states  and  counties 
are. 

So  when  you  wonder  what  kind  of 
accountability  information  the 
“feds”  want,  why  and  what  will  be 
of  use  to  them,  you  probably 
already  are  closer  to  the  answer 
than  you  realize.  Ask  yourself: 

•  Who  are  the  influentials  and 
decisionmakers  who  “call  the 
shots”  in  my  state  or  county? 

•  What  kind  of  questions  do  they 
ask  me  and  what  information  do 
they  want? 

•  What  kind  of  information  packag¬ 
ing  gets  their  attention,  approval 
and  action? 

You  will  shape  county 
accountability  and  evaluation  to 
respond  to  locally  identified 
problems,  issues  and  questions.  You 
will  decide  what  information  you 
need,  for  whom,  and  search  out 
how  to  find  it.  Your  progress 
reports,  conclusions  and  recom¬ 
mendations  will  get  passed  along  to 
supervisors,  local  influentials  and 
decisionmakers  and  interested 
others. 

Can  Strengthen  State  Reports 

When  your  CES  state  office  is  asked 
program  questions  or  must  justify 
budgets,  your  county  reports  will 
feed  into  whatever  response,  pro¬ 
posals  or  reports  are  developed. 
When  your  county  results 
contribute  data  which  can  be 
aggregated  with  that  of  other 
counties,  the  result  will  be  a  more 
powerful  total  state  report  than 
would  otherwise  be  possible.  Then, 


too,  occasionally  your  county  narra¬ 
tive  may  be  unique  or  so  outstand¬ 
ing  that  it  surfaces  as  a  separately 
identifiable  item  in  the  state  report. 

How  does  information  passed  along 
to  the  state  CES  office  get  used  in  the 
county?  Do  reports  outline  progress 
with  clientele,  improvements  or 
innovations  in  the  program 
development  process,  functional 
operation,  staffing  and  so  on?  Does 
the  budget  information  you  submit 
also  guide  your  review  of  the 
resource  allocations  within  your 
control  in  the  county?  Can  it  be 
used  to  justify  increases  in  funding, 
both  in-house  and  from  non- 
Extension  granting  sources?  The 
kind  of  information  most  useful  to 
you  in  the  county  is  probably  also 
most  useful  to  your  state  CES  office 
and  to  us  here  in  ES-USDA. 

One  fact  that  county  Extension 
workers  sometimes  forget  is  that 
CES,  the  state  partner,  is  the  channel 
through  which  the  county  partner’s 
accountability  and  evaluation 
information  reaches  ES-USDA. 
Information  provided  by  counties  is 
aggregated,  summarized,  combined 
or  otherwise  massaged  in  the  state 
level  reporting  process.  This  process 
often  leads  to  alteration  of  county 
reports.  But  this  is  to  be  expected  in 
a  process  which  takes  information 
useful  “as  is”  in  the  county 
generating  it,  and  reworks  it  along 
with  information  from  other 
counties  in  the  state  to  increase  the 
size,  scope  and  result  impacts  for 
the  state  as  a  whole.  It  is  this  larger 
in-impact  more  concise-in-size 
version  that  ultimately  reaches  us  in 
ES-USDA. 

The  federal  partner  goes  through 
state  accountability  and  evaluation 
information  in  a  parallel  manner  to 
that  of  states. 

What  Kind  of  Accountability 
and  Evaluation? 

That  which  helps  us  justify  budget 
requests,  respond  to  inquiries  from 
the  Congress,  congressional  aides, 
the  Government  Accounting  Office, 
U.S.  Office  of  Management  and 
Budget,  USDA  Secretary’s  Office 
and  others  at  the  Department  level, 
the  Assistant  Secretary  for  Science 
and  Education,  Extension  Admin¬ 
istrators  and  others.  State  Extension 


Directors  and  state  counterparts  to 
national  program  leaders, 
universities,  media,  agencies  and 
organizations,  commodity  groups, 
special  and  public  interest  groups, 
professional  and  trade  associations 
and  the  general  public. 

Response  to  Inquiries 

Information  counties  provide  to 
states  helps  us  convey: 

•  how  funds  are  being  used 

•  cost  effectiveness 

•  who  we  serve,  how  many,  how 

•  why  program  is  needed,  what’s 
happening,  accomplishments 

•  how  Extension  supports  USDA 
thrusts 

•  trends,  new  facets  of  existing 
programs 

•  specific  examples  of  program 
success,  documented  impact 

Who  Needs  The  Information? 

We  all  do,  in  part.  Individuals  use 
what  they  need  to  meet  the 
responsibilities  associated  with  their 
position. 

How  Is  The  Information  Used? 

In  many  ways.  Some  ES  staff  develop 
the  national  reports  required  by 
funding  sources  and  others  plan 
new  budget  justifications  and  pro¬ 
grams,  help  improve  existing  ones, 
generate  communication  to  states, 
and  territories,  prepare  special 
reports,  write  speeches  and  briefing 
memorandums  for  the  Secretary 
and  USDA  Administrators, 
create  media  releases,  serve  as 
liaison  to  the  groups  previously 
mentioned,  serve  as  a  locator 
reference  for  what  is  being  done 
where,  act  as  a  key  part  of  many 
networks,  and  more. 

The  accountability  and 
evaluation  information  you  provide 
to  document  local  county  happen¬ 
ings  ultimately  becomes  an  integral ' 
part  of  the  proCid  organizational 
achievement  record  all  of  us  in  the 
Extension  “family”  collectively  share 
as  partners.  □ 
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Guidelines  from  the 
Grassroots 


Remedy  for  Stored 
Grai'  orage 

The  Performance 
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Energy  for  Oregon — An 
Evaluation  Effort 

Resident  Program  Aides 
Make  a  Difference 


Programing  Tomorrow's 
Tourists 


Tennesse  Turns  to  TIS 


Which  Comes  First? — 
Problems  Or  Programs? 

The  Limits  of  Evaluation 


Documenting  Indicators 


Obtaining  Data 


RAP  Evaluates  Inservice 
Education 


4-H  Priority  Programs 
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Guidelines  From  The  Grassroots 


Jim  Shaner 

State  Information  Extension  Specialist 
University  of  Missouri,  Columbia 


They  were  animated  people,  most  in 
seed  caps  and  jeans.  Their  questions 
were  tough;  How  do  you  stop  tons 
of  soil  from  moving  off  your  farm? 
How  can  you  afford  to  save  soil 
when  you’re  plowing  every  acre  to 
turn  another  dollar  to  save  your 
farm?  Across  the  cavernous  room, 
educators  and  parents  listen  as  a 
minister  tells  of  a  single  mother — 
crushed  by  utility  bills  and  food 
costs — trying  to  cope  with  problem 
kids.  Different  members  of  the 
group  ask:  Where  will  tomorrow’s 
jobs  be?  Who  will  help  the  poor? 
The  elderly?  Together  they  turned 
the  Independence  Extension  Center 
into  a  warehouse  of  ideas  for  coping 
with  this  decade  of  change. 

Volunteer  Investment 

More  than  200  of  these  volunteers 
invested  roughly  2,600  hours 
meeting  and  studying  materials  to 
help  establish  Extension’s  educa¬ 
tional  priorities  and  approaches  in 
meeting  them.  Their  goal,  says 
regional  Extension  director  Al  Black, 
was  to  put  Kansas  City  Extension  “at 
the  cutting  edge”  for  developing 
educational  solutions  for  the 
problems  of  the  1980’s. 

To  ensure  substantive  results.  Exten¬ 
sion  used  a  four-step  process, 
developed  by  rural  sociologist  Daryl 
Hobbs,  University  of  Missouri  at 
Columbia,  and  Black.  First, 
interviews  of  60  opinion  leaders 
helped  establish  the  issues.  Then 
Extension  councils  in  each  county 
listed  the  issues  they  and  their 
neighbors  faced.  Next,  Kansas  City’s 
Extension  staff  collected  data  to 
analyze  trends.  Then,  using  the 
results  of  this  process,  volunteers, 
chosen  to  represent  the  broad 
interests  of  Kansas  City,  identified 
five  categories  of  concern: 
economic  development,  energy, 
family,  food  and  fiber,  and 
providing  human  services. 

New  things  are  already  happening 
in  Kansas  City.  For  one,  planning 


brought  together  groups  that  had 
never  been  together  before. 

That  led  to  the  “first  ever”  BEST 
Conference — Business  and 
Economic  Strategies  for  Teens.  The 
program,  cosponsored  by  the 
Missouri  Council  of  Economic 
Education  with  support  from  the 
Cray  Foundation,  used  a  quiz-bowl 
format.  It  taught  about  the  economy 
to  a  “cross  section  of  young 
people.”  Black  says  his  staff  also  is 
working  with  the  University  of 
Missouri  at  Kansas  City  and  others 
to  study  the  job  skills  that  will  be 
needed  for  tomorrow’s  work.  Then 
Kansas  City  Extension  will  work  to 
help  local  organizations  restructure 
their  curricula  to  provide  that 
training. 

Saving  energy  also  ranked  high  in 
Kansas  City.  Gail  Imig,  associate 
dean  and  director  of  Extension 
home  economics,  says  retrofitting 
makes  older  homes  more  efficient. 
“Retrofitting”  can  range  from 
insulating  windows  to  adding 
greenhouses  “to  try  to  make  passive 
solar  heating  more  functional.”  The 
first  step,  says  Imig,  is  an  energy 
audit  to  identify  “heat  loss  zones.” 

Glenda  Kinder,  area  specialist,  then 
developed  an  energy  audit  booklet 
and  a  videotape  for  airing  over  local 
cable  television.  It  “walks  people 
through”  an  energy  audit  of  two 
older  homes.  The  UMKC  Com¬ 
munications  Department  produced 
the  video.  After  training  people  to 
do  their  own  audits,  Kansas  City 
specialists  plan  to  train  them  to 
insulate  and  retrofit  their  own 
homes.  Plans  call  for  as  may  as  three 
more  teaching  programs  for  cable 
television. 

Early  Planners  for  the  Future 

This  kind  of  formal  planning  isn’t 
new  to  Missouri.  The  state’s  first 
published  area  plan,  completed  in 
southeast  Missouri’s  Bootheel,  dates 


Extension  is  helping  authorities  to  involve  citi¬ 
zens  in  the  fight  to  curb  crime.  Here,  at  a  special 
anticrime  exhibit  a  police  officer  displays  a  case 
with  drug  paraphernalia. 


back  to  1966.  Since  that  time  more 
than  30  plans  were  developed  for 
the  state’s  21  geographic  areas. 


In  the  first,  Bootheel  farmers  set 
Extension’s  directions  on  growing 
cotton.  Now  Extension  planning 
encompasses  Missouri’s  broad  range 
of  expertise.  Besides  such  traditional 
disciplines  as  agriculture,  the  Show- 
Me  State  offers  field  specialists  in 
continuing  education  for  profes¬ 
sionals,  business  and  industry 
development,  and  in  community 
development  and  local  government. 
Specialists  in  those  last  two  dis¬ 
ciplines  can  assist  communities  in 
doing  self-study  surveys  that  aid  in 
the  planning  process.  All  are  part  of 
Missouri’s  Cooperative  Extension 
network,  says  Director  Leonard 
Douglas. 

Naturally,  Douglas  sees  the  purpose 
of  these  plans  as  drawing  direction 
from  the  people  they  serve. 


“You  have  to  get  to  people  talking 
about  their  own  counties”  for  the 
planning  process  to  work,  says 
Douglas.  That  means  tangible  goals 
that  people  can  achieve.  If  a  goal  is 
identified  “by  the  people  them¬ 
selves,  it’s  identified  in  terms  of 
what  it  takes  to  reach  it.  It  brings 
commitment.” 
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Kansas  City  4-H'ers  in  a  teen  program 
learn  first-hanrt  about  careers  in  medicine.  At 
the  Lakeside  hospital  in  south  Kansas  City  an 
emergency  room  nurse  chats  with  the  youth 
about  her  various  duties. 


Top;  Skyline  of  Kansas  City  reveals  trucks  loaded  with  produce  moving  past  elevators 
along  the  Missouri  River.  Above:  Crain  prices  are  scrutinued  by  brokers  at  the  Kansas 
City  Board  of  Trade. 


Nationwide  Extension  uses  grass¬ 
roots  direction  for  its  programs.  In 
Missouri  those  local  results  can  have 
a  statewide  impact — and  vice 
versa — as  Extension  staff  work  to 
integrate  state  and  area  plans. 

Kansas  City  Planners 

In  Kansas  City  that  process  was 
rather  straightforward.  Tom  Hill, 
director  of  4-H  youth  programs  for 
Missouri,  met  with  Black  and  the 
region’s  youth  staff  to  mesh  their 
local  plans  with  two  national 
one’s — Extension  in  the  1980’s  and 
the  national  needs  assessment  done 
for  4-H  by  North  Carolina  State 
University.  Tom  Henderson, 
director  of  Misouri’s  business  and 
industry  program,  says  Kansas  City’s 
planning  resulted  in  a  “nearly 
perfect  mesh”  between  the 
economic  concerns  there  and  his 
program’s  major  thrusts. 

For  instance,  a  $75,000  grant  from 
the  Economic  Development  Admin¬ 


istration  helped  extend  the 
individual  counseling  Extension 
offers  Missouri  business  owners.  In 
Kansas  City  alone,  Henderson 
reports  that  brought  in  roughly 
$115,000  in  new  investment  and  the 
program  saved  two  jobs  and  created 
four  more.  Projections  for  the  state 
include  more  than  400  jobs  saved, 
more  than  250  new  ones  created, 
and  new  investments  tallying  as 
much  as  $8V4  million. 

Advisory  Committees 

However,  not  all  direction  for 
Extension  in  Missouri  comes  from 
area  plans.  Advisory  committees, 
beginning  with  those  that  work  with 
Douglas  and  with  the  university 
system  president,  James  C.  Olson, 
also  play  a  major  role. 

Advisory  committees  also  help  set 
directions  in  home  economics, 
youth,  agriculture,  and  in 
Henderson’s  business  and  industry 
and  continuing  education  programs. 
Perhaps  the  most  structured  of 


those  approaches  is  used  in  home 
economics.  Gail  Imig  explains: 
“Planning  starts  at  the  county  level. 
Advisory  committees  work  with 
each  Extension  home  economist  to 
identify  local  needs.  These  are 
funneled  through  Extension’s  field 
staff  representatives  who  identify 
key  needs  across  the  state  for  their 
corresponding  academic  depart¬ 
ments.  Teleconferencing,  first 
within  a  region  to  give  the  local 
representative  a  broad  perspective 
and  then  among  the  areas  and 
regions,  helps  supply  background  to 
these  representatives  so  they  can  set 
priorities  at  a  campus  meeting  each 
December.”  (A  “region”  usually  is 
comprised  of  two  or  three  Extension 
“areas.”  Each  area  might  include 
from  three  to  10  counties.) 

Local  planning  and  needs  are  taken 
into  account  in  those  first  meetings 
and  throughout  the  process.  Then 
Imig  works  to  integrate  all  this  plan¬ 
ning  into  a  master  plan  that  takes 
Extension’s  national  plan  into 
account.  Members  from  each  of  the 
program  advisory  committees  sit  on 
an  overall  committee  together  with 
a  representative  for  the  Extension 
Homemakers  Program  and  one  for 
the  Expanded  Food  and  Nutrition 
Program. 
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Below:  Improving  reproductive  efficiettcy  in  hog 
and  livestock  production  is  an  important  part  of 
Extension’s  commodity  planning  efforts.  Here, 
swine  producers  lim  and  Lois  Phillips  of  Drexel, 
Missouri,  get  tips  on  the  latest  equipment  to 
intprove  their  weekly  farrowing  schedule. 


Together,  they  integrate  national 
trends  and  guidelines  with  what  the 
staff  knows  about  Missouri  families 
and  the  latest  research  findings. 

Then  Imig  says  they  “negotiate”  the 
thrusts  of  their  statewide  home 
economics  program  for  the  year. 
Area  staff  adapt  them  to  their  areas 
and  regions. 

Other  Planning  Efforts 

Besides  the  area  planning  process 
used  by  all  Extension,  the  college  of 
agriculture  gets  direction  from  two 
other  planning  efforts.  Charles  E. 
Campbell,  assistant  director  for  agri¬ 
cultural  Extension,  led  in  one — 
commodity  planning. 

Because  that  planning  concentrates 
on  a  commodity  on  a  one-time 
basis,  Campbell  observes,  “You  get 
involvement  from  leadership  who 
won’t  participate  in  the  more  long¬ 
term  approach  of  Extension 
councils.  Therefore,  you  get  some 
ideas  you  wouldn’t  get  otherwise.” 
Reaching  those  new  people  was 
“one  of  the  satisfactions  for  me,” 
Campbell  says. 

The  purpose  of  the  commodities 
planning  effort  is  to  identify 
research  problems.  Still,  Campbell 
says.  Extension  plays  a  strong  role. 
Field  specialists  worked  to  make 
people  aware  of  the  hearings  and  to 
encourage  attendance.  An  Exten¬ 
sion  specialist  convened  every  com¬ 
modity  hearing.  University 
researchers  provided  situation 
statements.  Then  an  audience. 


totaling  roughly  1,600  Missourians 
over  n  locations,  responded  with 
their  needs  and  priorities. 

They  covered  14  commodities, 
ranging  from  such  traditional  ones 
as  corn  and  beef  to  horses,  cotton, 
rice  and  tobacco. 

The  other  planning  approach, 

“Food  for  the  21st  Century,” 
involved  lengthy  discussions  among 
university  social  and  biological 
scientists  and  scientists  in  the  agri¬ 
business  industry.  The  scientists 
agreed  that  much  current  research 
just  shows  that  things  do  happen. 

But  to  make  further  progress  in 
plant  and  animal  growth  they 
needed  to  learn  more  about  why 
and  how  things  happen. 

Consequently,  for  long-term  plan¬ 
ning,  the  scientists  agreed  the 
university  should  deal  with  “very 
basic  research  and  abstract  knowl¬ 
edge  needs.”  They  cited  “the 
genetic  manipulation  of  plants  and 
animals.”  But  even  that  ties  in  with 
the  commodities  planning  process. 
Livestock  producers  also  wanted  to 
improve  reproductive  efficiency.  So 
Campbell  concludes  the  two  plans 
“were  not  totally  discrete.” 

Why  should  Extension  be  involved 
in  setting  research  directions? 
Sanders,  associate  dean  and  director 
of  agricultural  Extension,  observes 
that  Extension  involvement  in  both 
these  research  planning  efforts  fits 
well  with  the  area  specialist’s  role. 


•J  ■  I  *  '■ 
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Top:  Extension  in  Missouri  has  played  a  major 
role  in  assisting  neighborhoods  to  rehabilitate 
housing.  Above:  Local  parks  have  been  the  sites 
of  successful  crime  watch  programs. 


“That’s  what  makes  Extension 
unique,  it  delivers  research-based 
information,”  he  says. 

However  good  Missouri’s  area 
planning  system  is,  it’s  not 
foolproof.  Both  Douglas  and 
Sanders  would  like  to  do  more  to 
get  the  directions  set  in  area 
planning  worked  into  Missouri’s 
statewide  structure. 

“Locally,  they  can  plan  their 
programs  fairly  well,”  Sanders  says, 
“because  we  have  area  specialists 
with  great  technical  competency. 

But  the  weakness  appears  to  be  the 
translation  of  the  plan  to  the  state 
specialist.” 

The  director  of  agricultural 
Extension  continues,  “At  least  there 
doesn’t  seem  to  be  the  direct 
mechanism  to  make  sure  that  the 
flows  of  information  come  together. 
We  would  be  remiss,  if  we  didn’t 
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Martin  Phillips  uses  a  personal  computer  to  help 
the  family  firm  keep  track  of  the  8,000  hogs 
marketed  each  year. 


consider  the  local  plans.”  Sanders 
continues,  “If  the  project  in  animal 
science  is  going  to  be  able  to  assist, 
there  needs  to  be  the  opportunity 
for  flow  of  that  information  back 
and  forth.” 

Problems  and  Solutions 

Another  problem  is  tight  funding — a 
problem  faced  by  Extension 
everywhere.  The  state’s  21  areas 
have  been  going  through  a 
grassroots  planning  process  every  5 
years.  That  means  four  or  five  plans 
every  year.  To  cut  down  on  that 
burden  and  on  other  administrative 
costs,  Douglas  is  moving  Missouri 
Extension  to  a  nine-region  structure. 
County  and  area  planning  would 
continue,  Douglas  says,  but  formal, 
5-year  planning  would  be  done  by 
an  entire  region. 

All  this  planning  generates  another 
concern  for  Sanders — that  the 


planning  document  itself  might 
become  an  edict.  “Planning  is  a 
process  of  self  examination,”  he 
says,  “and  the  documentation  that 
results  from  that  can  be  a  guide,  but 
not  the  sole  basis  on  which  you 
perform.  It  has  to  be  flexible.”  If  it’s 
inflexible,  Sanders  says,  “you’re  a 
victim  of  the  plan.” 

So  he  would  like  to  see  the  area 
plans  informally  updated  every  year. 
The  former  field  entomologist 
believes  that  would  make  them 
more  flexible  and  keep  staffs  tuned 
to  their  major  directions  and  to  cur¬ 
rent  conditions.  Yearly  updates  also 
would  make  formal  documentation 
and  modification  of  a  regional  plan, 
every  5  years,  “a  fairly  easy 
proposition.”  Sanders  suggests 
letting  “the  planning  process 
generate  the  thoughts,  the  ideas, 
the  innovations”  and  following 
them  “as  closely  as  you  possibly 
can,”  but  altering  that  plan  as  things 
change. 

Does  all  this  planning  work? 

Well  it  seems  to,  but  not  always  the 
same  way.  The  smaller  staff  working 
in  the  business  and  industry  and  in 
the  continuing  education  programs 
precludes  those  specialists  from 
meeting  with  advisory  committees 
in  every  county.  So  Henderson’s 
continuing  education  specialists  use 
a  series  of  ad-hoc,  or  “self  destruct- 
ing”  committees.  They  meet  to  set 


Missouri  livestock  producers  such  as  Charles 
Kinney  of  Liberty  Bell  Farm  appreciate  the  bene¬ 
fits  of  Extension's  research-based  information. 
Kinney,  (right)  president  of  Missouri's  Red  Angus 
Association,  proudly  shows  off  a  prize-winning 
bull. 

up  programs  by  a  subject,  such  as 
health  care,  and  then  stop  meeting 
once  a  program  is  in  place  to  meet 
those  needs. 

John  Henschke  used  that  approach 
to  organize  a  five-session  course  on 
sex  education.  It  reached  more  than 
250  parents  and  their  children  in 
Maryville  and  in  Albany  in 
northwest  Missouri.  The  continuing 
education  specialist  worked  with 
child  and  family  development 
specialist  Diana  Bonner  to  defuse 
this  potentially  explosive  topic  and 
set  up  the  course. 

“The  key  to  success  is  offering  such 
a  course  was  making  the  program 
noncontroversial  by  building  a 
foundation  of  endorsement  and 
support  within  the  community,” 
Henschke  observed.  To  do  that, 
Henschke  and  Bonner  contacted 
teachers,  school  administrators,  the 
Nodaway  County  Ministerial  Fellow¬ 
ship  and  others.  Then  the  two 
Extension  specialists  sought  the 
endorsement  of  local  civic  groups 
by  describing  the  program’s 
approach  and  value. 

“The  basic  concept  behind  the 
course  is  to  make  the  parent  the 
primary  source  for  human  sexuality 
information,”  said  course  instructor 
jean  Brown,  Family  Guidance 
Center  at  St.  Joseph. 

A  train-the-trainer  approach  added 
76  new  instructors  to  the  St.  Joseph 
program.  Last  spring  they  reached 
other  communities  in  northwestern 
Missouri  and  nearby  Iowa  with  it. 
And  the  national  Public  Broadcast¬ 
ing  System  will  televise  highlights 
early  in  1984  as  part  of  a  seven-part 
series  called  “Your  Child,  My 
Children.” 

The  idea  of  moving  government 
decisionmaking  back  to  the  people 
caught  on  with  the  Extension 
Service— nationally — decades  ago. 
Missouri  just  shows  how  much.  □ 
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Remedy  for  Stored  Grain  Spoilage 

Marlene  Fritz 

Agricultural  Information  Specialist,  CES 
University  of  Idaho 


Five  years  ago,  the  Idaho  Feed  and 
Grain  Association  approached  the 
University  of  Idaho  College  of 
Agriculture  with  concerns  about 
stored  grain  spoilage.  Growers  in 
southeastern  Idaho  were  losing  up 
to  5  percent  of  their  crop — $10 
million  annually. 

To  be  responsive  to  their  clientele,  a 
Ul  research  agricultural  engineer 
and  an  Extension  entomologist  in  a 
3-year  research  project  showed  that 
aeration  could  eliminate  nearly  all 
stored  grain  losses  in  the  region. 

Nor  would  costly,  and  often  inef¬ 
fective,  chemical  grain  protectants 
be  necessary. 

Demonstration  Bins 

This  year.  University  of  Idaho  staff 
made  5  grain-filled  demonstration 
bins  available  and  electronically 
monitored  aeration  systems  were 
provided  by  project  funds. 

Cooperator  Donald  Trupp  of  New- 
dale,  involved  in  the  project  since 
its  inception  says,  “We  were  looking 
at  this  sort  of  thing  5  or  6  years  ago 
because  we  wanted  to  know  what 
was  going  on.  We  knew  we  were 
having  trouble. 

“The  Ul  people  have  done  a  good 
job  and  have  been  very  receptive  to 
our  needs.’’ 

Broad  Based  Support 

The  project  has  attracted  broad 
support  within  the  university  and 
the  client  groups  Extension  serves. 
According  to  Ray  Prigge,  director  of 
agricultural  programs  in  south¬ 
eastern  Idaho  for  the  Ul  College  of 
Agriculture:  “We’ve  drawn  profes¬ 
sionals  from  across  functional  lines 
to  develop  a  consensus  of  need  and 
a  program  of  action. 

“Very  significant  is  the  involvement 
of  four  farmers,  a  researcher,  two 
Extension  specialists,  seven  county 


agents,  a  district  director,  the  Great 
Western  Malting  Co.,  Anheuser- 
Busch,  Coors,  the  Idaho  Wheat 
Growers,  the  Idaho  Wheat  Commis¬ 
sion,  county  wheat  growers’  groups, 
county  commissioners,  a  com¬ 
mercial  grain  elevator,  local  agri¬ 
business  firms  and  USDA  Integrated 
Pest  Management  staff.’’ 

Prigge  hopes  that,  after  successful 
demonstration,  evaluation,  and 
adoption  by  southwestern  Idaho 
clientele,  the  program  will  be  used 
elsewhere  in  the  country. 

Lynn  Murdock,  Bingham  County 
cooperator,  reports  that  his  friends 
and  neighbors  are  showing  increas¬ 
ing  interest.  “Nobody  is  ready  to 
spend  the  money  if  it’s  just  some¬ 
thing  to  keep  them  from  climb¬ 
ing  up  the  outside  of  the  bin.  But 
once  they  find  out  that  the 
technology  can  do  something  for 
them — like  keep  insects  out  and 
raise  their  profits — then  they’ll  do 
it,’’  he  says. 

Series  of  Meetings  Lead  to  Project 

The  research  project  resulted  from  a 
series  of  meetings  across  southeast¬ 
ern  Idaho  in  1978  at  which  growers, 
field  representatives,  elevator 
operators,  and  county  agents  heard 
technical  presentations  and  dis¬ 
cussed  the  problem.  They  agreed 
that  Ul  scientists  should  probe  the 
extent  of  the  losses,  their  causes, 
and  their  remedies. 

By  1981,  ag  engineer  James  Halder- 
son  and  Extension  entomologist 
Larry  Sandvol  had  determined  that 
USDA  recommendations  favoring 
chemical  grain  protectants  did  not 
apply. 

The  region’s  cool  nights — generally 
falling  below  the  60-degree 
Fahrenheit  minimum  required  for 
egg-laying  by  most  insects — 
eliminated  the  threat  of  high  insect 
populations  when  aeration  systems 
were  conscientiously  used. 


Halderson  estimates  that  fewer  than 
1  in  10  southern  Idaho  grain 
growers  used  aeration  systems  while 
“substantial  numbers’’  applied  grain 
protectants.  The  initial  cost  of  elec¬ 
tronically  monitored  aeration 
systems  is  $400  to  $1,500,  and  they 
are  inexpensive  to  operate. 

Halderson  and  Sandvol  found  that 
grain  protectants  actually  may  con¬ 
tribute  to  storage  losses  by  giving  a 
false  sense  of  security. 

“The  top  center  part  of  the  bin  is 
where  you’re  most  likely  to  find 
grain  insects  during  the  colder 
winter  months,  and  that’s  where 
heat  and  moisture  tend  to  break 
down  the  grain  protectants,’’ 
Halderson  says. 
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When  the  research  was  completed,  of  $1,000  for  the  demonstration 
Halderson,  Sandvol,  and  Prigge —  came  from  the  Anheuser-Busch 

along  with  agricultural  economist  Malt  Barley  Growers  Association. 
Joseph  Guenthner  and  county  U.S.  Integrated  Pest  Management 

agents  Brian  Finnigan,  Sterling  funds  of  $22,000  were  obtained 

Schow,  Cecil  Alldaffer  and  James  through  U1  IPM  specialist  Edward 

Whitmore — met  to  weigh  ideas  on  Bechinski. 
how  to  expedite  technology  trans¬ 
fer.  Settling  on  farm-based  In  addition  to  providing  bin 

demonstrations,  they  proposed  the  instrumentation  and  staff  travel,  the 
concept  to  industry  groups  and  funds  will  be  used  to  develop 

clientele  in  November  and  educational  packages,  including 

December  1982.  seven  publications  in  the  Current 

Information  Series,  slide-tape  sets,  a 
Positive  Response  20-minute  public  service  TV 

The  response  was  overwhelmingly  program,  and  5  to  10  public  service 
positive.  All  key  groups  declared  the  announcements, 
educational  program  to  be  a  high- 

priority  need.  On  May  10  and  12, 1983,  a  20- 

minute  videotape  on  the  grain  bin 

Local  agribusiness  firms  volunteered  demonstration  was  broadcast  on 
equipment  and  installation  services.  Pocatello  television. 

Four  local  farmers  and  one  com¬ 
mercial  grain  elevator  owner  Growers  saw  the  project  this  summer 

donated  the  use  of  their  round,  through  tours  led  by  county  agents, 

steel  bins  for  instrumentation  and  One  of  these  tours  was  sponsored 
demonstration.  In  addition  to  by  the  National  Association  of 

Murdock  and  Trupp,  the  co-  Wheat  Growers, 

operators  include  Doyal  Stiles  of 

Caribou  County,  Kenneth  Koompin  “This  is  what  we  had  in  mind,”  said 

of  Power  County  and  David  Reinke  cooperator  Trupp.  “It’s  a  really 

of  Fremont  County.  good  example  of  working  toward  a 

common  goal.  And  an  example  of 

Anheuser-Busch,  Coors,  and  Great  accountability  to  users  and  ongoing 
Western  Malting  Co.  provided  evaluation  in  action.”  □ 

$9,000  for  the  research.  Seed  money 


Spoilage  of  stored  grain  in  southeastern  Idaho 
totals  about  $10  million  in  losses  to  growers 
annually.  The  remedy  appears  to  be  aeration.  A 
3~year  research  project  by  the  University  of 
Idaho  College  of  Agriculture,  involving  Exten¬ 
sion  specialists  and  professionals  from  many  dis¬ 
ciplines,  showed  that  aeration  systems  are  vastly 
more  successful  cutting  grain  losses  than  costly 
chemical  grain  protectants.  The  project  used  5 
grain-filled  demonstration  bins  that  were  elec¬ 
tronically  monitored. 
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The  Performance 
Appraisal  Connection 

Peter  Warnock 

Extension  District  Director 

University  of  Florida,  Gainesville 


Both  agent  performance  appraisal 
and  program  evaluation  attempt  to 
measure  the  success  of  Extension 
work.  One  assesses  process,  the 
other  the  finished  product. 
Extension  programs  are  to  an  agent 
what  a  cake  is  to  a  cook  or  a  house 
to  a  carpenter.  For  these  profes¬ 
sionals  to  evaluate  process  without 
evaluating  the  product  would  be 
woefully  inadequate,  leaving  them 
with  too  many  unanswered 
questions. 

Florida  Cooperative  Extension  was 
deeply  involved  in  strengthening 
their  program  planning  and 
evaluation  practices  when  two 
closely  related  occurrences  added 
impetus  to  their  efforts.  Roger 
Neiswinder,  Seminole  County  Ad¬ 


ministrator,  laid  out  the  challenge  to 
Extension  faculty  in  1981  when  he 
asked  them  to  account  for  their 
actions  by  answering  three 
questions:  What  are  you  doing? 

How  are  you  doing  it?  What 
difference  are  you  making? 

Questions  1  and  2  were  relatively 
easily  answered,  whereas  little  could 
be  said  in  response  to  question  3. 

At  about  the  same  time,  motivation 
to  change  came  from  a  state 
legislative  oversight  committee, 
which  probed  deeply  into  the  pro¬ 
grams,  practices  and  policies  of  the 
University  of  Florida’s  Institute  of 
Food  and  Agricultural  Sciences.  Like 
Neiswinder,  state  legislators  also 
were  asking  the  question — “What 
difference  are  you  making?” 


Means/Ends  Dichotomy 

Extension  program  evaluation  and 
personnel  performance  appraisals 
have  always  been  closely  related.  As 
Extension  agents  know,  to  conduct  a 
comprehensive  and  meaningful 
program  review  without  addressing 
agent  performance  is  almost 
impossible. 

By  and  large,  internal  managers,  like 
Extension  agents,  cooks,  and 
carpenters  continue  to  be  con¬ 
cerned  with  both  ends  and  means. 
Competition  for  tax  dollars, 
however,  has  made  the  ends  more 
important  to  external  evaluators. 
These  judges  are  like  the 
restaurant’s  customer  who  judges 
the  cake  by  its  appearance,  taste,  and 
cost  or  the  home  buyer  who 
analyzes  a  home  purchase  on  the 
basis  of  its  utility,  attractiveness, 
cost,  and  projected  resale.  The 
means  used  by  the  cook  and 
carpenter  are  of  little  concern  to  the 
buyer  of  these  products.  Because  of 
their  limited  time  and  expertise, 
they  are  primarily  interested  in  the 
final  product  and  its  relative  value. 

Means  to  the  Extension  agent 
provide  internal  measures  necessary 
to  improve  program  performance. 

In  the  mid-1970’s,  the  American 
Institute  for  Research,  under 
contract  with  the  Extension  Service, 
USDA,  established  that  successful 
Extension  agents  are  proficient  in 
program  planning,  program 
promotion  and  public  relations, 
program  implementation,  gaining 
program  support,  interper¬ 
sonal/personal  relations  and 
supervision.  Only  through  careful 
attention  to  these  means  of  doing 
business  can  new  and  experienced 
agents  hope  to  achieve  excellence 
in  their  work.  They,  like  superior 
cooks  and  carpenters,  must  be  able 
to  understand  and  evaluate  both  the 
means  and  ends  which  affect  their 
work. 
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Agent  Expectations 

Making  the  intellectual  shift  in 
program  evaluation  and  per¬ 
formance  appraisal  from  means  to 
program  ends  is  not  easy.  The 
National  Accountability  and  Evalua¬ 
tion  System  (A/E)  facilitates 
emphasis  on  final  product  as 
reflected  in  4-year  major  programs. 
Many  Florida  faculty  are  optimistic 
about  potential  benefits  to  them 
and  their  clients.  Following  are  a 
few  of  their  solicited  expectations: 

Marion  County  home  economics 
program  leader  jenta  Wyatt  believes 
that  agents  will  gain  increased  satis¬ 
faction  and  recognition  from  4-year 
major  programs  as  compared  to 
annually  planned  programs  of  lesser 
consequences. 

Ryan — district  director — is  con¬ 
vinced  that  Extension’s  potential 
survival  in  Florida  depends  on  effec¬ 
tively  measuring  and  reporting 
clientele  benefits  in  specific 
measurable  terms. 

“The  new  A/E  system  maintains 
individual  expression.  Extension 
program  development  is  still  a 
bottom-up  planning  process  which 
begins  at  the  county  level,”  says 
Jesse  Godbold,  Clay  County 
Extension  director.  “State  and 
national  data  collection  and  analyses 
could  help  Florida  agents  do  a 
better  job  by  seeing  what  others  are 
doing  with  similar  programs  around 
the  country.” 

“My  problem  has  always  been  not 
having  enough  time  to  adequately 
plan,  implement,  and  evaluate  pro¬ 
grams,”  says  Uday  Yadav,  Orange 
County  horticultural  agent.  “It 
appears  that  the  4-year  planning 
cycle  gives  us  more  time  to 
negotiate,  modify,  and  gather  data 
to  determine  with  confidence  if  we 
did  make  a  difference  in  the  lives 
and/or  businesses  of  our  clientele.” 


Marcia  Crocker,  4-H  program 
leader,  sees  4-year  program  plans  as 
a  means  of  putting  more  objectivity 
and  fairness  into  performance 
appraisals.  In  her  view,  “a  4-year 
major  program  requires  a  clear 
understanding  on  the  part  of  the 
agent  and  supervisor  as  to  what  will 
be  done;  how  it  will  be  dene;  and 
what  difference  it  will  make.” 

Barriers  to  Overcome 

The  new  A/E  system  could  fail  if 
agent  behavior  remains  unchanged. 
Habit  gives  predictability  to  work 
and  play  which  is  indispensable,  but 
it  can  also  seriously  weaken  an 
individual  or  organization  in 
transition.  The  challenge  facing 
Florida’s  agents,  specialists,  and 
volunteers  is  to  shift  attention  to  pro¬ 
gram  ends  by  spending  more  quality 
time  in  planning  and  evaluating 
Extension  programs.  This  change 
will  require  considerable  ego 
strength  and  self-discipline  since  it 
moves  counter  to  the  temperament 
of  most  agents.  For  example. 
Extension  agents  are  known  as 
“doers,”  who  enjoy  seeing  things 
happen!  Typically,  agents  prefer  to 
write,  teach,  demonstrate,  and 
counsel.  Four-year  major  programs 
undoubtedly  will  require  more  time 
and  effort  to  plan  and  evaluate  pro¬ 
grams,  probably  at  the  expense  of 
these  preferred  tasks  and  activities. 

Performance  appraisals  and  pro¬ 
gram  evaluations  can  be  emotional 
and  disheartening  experiences. 

With  few  exceptions,  adults  don’t 
like  to  be  reviewed,  analyzed, 
compared,  or  evaluated.  Therefore, 
the  new  A/E  system,  with  increased 
emphasis  on  evaluation  of  program 
results  (ends),  carries  with  it  a  threat 
to  one’s  professional  standing.  In 
this  era  of  accountability,  agents 
must  be  able  to  rise  above  these 
legitimate  concerns  by  using 
elements  of  the  A/E  system  which 
are  designed  to  provide  sufficient 


time  to  adequately  plan  and 
evaluate  programs,  using  objective 
and  fair  means,  while  maintaining 
individual  expression. 

Florida  Practice 

Florida  Cooperative  Extension  has 
anticipated  the  current  emphasis  on 
program  ends  in  its  program  evalua¬ 
tion  and  performance  appraisal 
practices.  For  the  past  two  program 
years,  agents  have  been  primarily 
appraised  on  the  basis  of  measured 
changes  in  client  knowledge,  skills, 
attitudes,  aspirations,  practices, 
and/or  behaviors.  Faculty  from  the 
Program  Evaluation  and  Organiza¬ 
tional  Development  Unit,  along 
with  program  deans  and  district 
directors  have  conducted  extensive 
training  on  planning,  evaluating, 
and  reporting  major  programs. 
Departmental  Extension  specialists. 
Extension  faculty  at  many  of  the 
state’s  23  research  centers,  and 
agents  received  this  instruction 
which  re-emphasizes  the 
importance  of  program  ends  in  the 
program  development  process. 
Effecting  change  in  the  professional 
ranks  is  difficult,  time  consuming, 
and  at  times  discouraging,  but  major 
substantive  programs  are  beginning 
to  appear  and  be  implemented. 

Program  means  and  ends  continue 
to  be  the  fundamental  components 
of  program  evaluation  and 
performance  appraisal.  Emphasis  in 
1984  is  on  the  question,  “What 
difference  are  you  making?”  Florida 
agents,  specialists,  and  volunteers 
are  preparing  to  answer  this 
question  through  their  4-year  major 
programs  and  performance 
appraisal  system  which  focuses  on 
program  ends.  There  is  a  sense  of 
optimism  that  they  will  be  able  to 
justify  their  existence  as  part  of  a 
national  educational  system  with 
proven  ability  to  adjust,  adapt,  and 
go  with  the  flow.  □ 
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Energy  For  Oregon — 
An  Evaluation  Effort 

Joyce  Patterson 

Extension  Communications  Specialist,  Energy 
Oregon  State  University,  Corvallis 


The  Oregon  State  University 
Extension  energy  program  was  new. 
And  most  of  the  energy  agents  and 
the  only  full-time  energy  specialist 
were  also  new  to  Extension.  It  was 
important  to  evaluate  our  efforts 
from  the  start — to  see  if  what  we 
were  doing  and  how  we  were  doing 
it  actually  helped  people  save 
energy. 

That  was  in  1980.  Now,  BVi  years 
later,  we  can  report  results  of  three 
major,  formal  evaluations  and  many 
less  formal  ones  along  the  way.  And 
we  have  five  more  formal  evalua¬ 
tions  in  process. 

It  was  important  for  professionals  to 
help  our  evaluation  efforts.  None  of 
us  knew  how  to  conduct  evalua¬ 
tions.  First,  we  hired  Com¬ 
munication  Design  in  Seattle  to  hold 
an  evaluation  workshop  for  us. 

Next,  we  contracted  with  the  Survey 
Research  Center  at  Oregon  State 
University  (OSU)  to  help  us  conduct 
three  major,  scientifically  designed 
evaluations.  Smaller  ongoing  evalua¬ 
tions  we  do  ourselves. 

What  have  we  learned?  The  answer 
to  this  question  can  be  divided  into 
two  parts;  What  we’ve  learned 
about  evaluation  and  what  we’ve 
learned  about  our  programs. 

Evaluation — Why? 

We  learned  that  evaluation  is  an 
attempt  to  determine  what  hap¬ 
pened  because  of  a  particular  pro¬ 
gram.  And  we  learned  that  building 
evaluation  into  a  program  from  the 
beginning  improves  that  program. 
The  evaluation  process  forces  us  to 
set  goals  and  to  plan  activities  that 
are  specific,  attainable,  measurable, 
and  pertinent  in  order  to  reach 
those  goals. 


Program  evaluation  has  many  forms. 
For  example,  take  an  Extension 
bulletin.  Before  anyone  on  the 
energy  faculty  writes  a  manuscript, 
we  discuss  together  the  need  for  a 
bulletin  on  that  topic,  availability  of 
similar  bulletins,  and  the  use  we  see 
for  the  bulletin.  Before  we  send  a 
manuscript  to  an  editor,  we  ask 
experts  in  the  appropriate  field  to 
review  it.  We  also  ask  other  Exten¬ 
sion  workers  to  review  it.  That’s  all 
part  of  evaluation. 

Publication  Evaluation 

For  a  more  formal,  scientific 
evaluation  of  a  specific  publication, 
we  contracted  with  the  OSU  Survey 
Research  Center.  Their  activities 
range  from  consultation  to  handling 
all  parts  of  a  survey. 

The  publication  evaluated  was 
written  by  two  of  our  energy  agents. 
Its  topic — solar  water  heating — 
included  an  explanation  of  different 
systems,  the  need  for  other  con¬ 
servation  measures,  and  consumer 
protection  information. 

From  questions  suggested  by  the 
energy  staff,  the  Survey  Research 
Center  developed  a  telephone 
questionnaire.  Readers  of  the 
publication  were  asked  to  evaluate 
its  relevance  and  helpfulness.  These 
readers  were  also  quizzed  about 
their  hot  water  energy  conservation 
awareness,  knowledge,  and 
behavior. 

Survey  results  indicated  that  9  out  of 
10  of  those  who  received  the 
publication  read  at  least  some  of  it; 
that  three-quarters  of  the  readers 
found  that  it  provided  the  kind  of 
information  they  expected  when 
they  requested  it,  that  it  helped  86 
percent  of  the  readers  make 
decisions  to  take  conservation 
measures,  and  that  7  percent  of  the 
readers  installed  a  solar  water 
heating  system  within  6  months 
following  receipt  of  the  publication. 


Workshop  Evaluation 

Our  workshops  deliver  energy 
conservation  information  to  the 
public.  At  the  end  of  each 
workshop,  participants  fill  out  a 
form  evaluating  that  paiticular 
session.  From  these  evaluations,  we 
learned  that  6-hour  workshops  are 
too  long.  Workshops  that  are  3  or  4- 
hours  long  are  better. 

The  Survey  Research  Center  also 
evaluated  one  series  of  workshops — 
those  conducted  in  the  Portland 
metropolitan  area  on  solar  water 
heating.  In  fact,  we  had  them  do 
two  evaluations  of  these  workshops. 
And  we’re  planning  a  third. 

Because  the  first  survey  found  that 
15  percent  of  the  first-year  partici¬ 
pants  installed  solar  water  heating 
within  the  year  and  75  percent  more 
intended  to  do  so,  we  did  a  follow¬ 
up  survey  the  next  year.  By  then,  23 
percent  had  installed  and  56  percent 
still  intended  to  do  so.  The  next 
followup  survey  is  targeted  at  this  56 
percent  to  see  if  they’ve  followed 
through  with  their  intentions. 

Telephone  Consultation  Evaluation 

Our  staff  spends  a  great  deal  of  time 
on  the  telephone  answering  energy- 
related  questions,  so  we  decided  to 
see  how  effective  this  delivery 
method  really  is. 

For  6  months  we  recorded  names, 
addresses,  the  nature  of  the  inquiry, 
and  our  response. 

The  Survey  Research  Center  helped 
us  develop  a  mail  questionnaire  that 
included  items  concerning  the  kinds 
of  conservation  measures  people 
called  about,  action  they  had  taken 
(and  when),  energy  savings  they  had 
realized,  the  type  of  information 
they  had  requested,  the  extent  to 
which  their  needs  for  information 
had  been  met,  how  helpful  the 
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information  had  been 
in  saving  energy,  and 
any  other  organizations 
they  had  contacted  for 
similar  help. 


Their  answers  showed 
that  of  the  people 
who  called,  about  60 
percent  had  taken 
some  conservation 
measure(s),  30 
percent  within  8 
months  following 
the  phone  call. 
About  two-thirds 
of  (hose  who  took 
conservation 
measures  noticed 
a  reduction  in  en¬ 
ergy  consumption. 
And  of  these,  73 
percent  found  the 
information  they 
received  from  Ex¬ 
tension  very  helpful 
in  enabling  them  to 
save  energy.  Another 
26  percent  found  the 
information  some 
what  helpful.  Fewer 
than  half  the  callers 
requested  similar  help 
or  information  from 
other  organizations 
or  agencies. 


This  response  was 
positive  enough  that 
our  staff  decided  it  is 
important  to 
continue  answering 
telephone  inquiries, 


These  three  major 
evaluations  cost 
about  $10,000, 
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with  funding  coming  from  a  U.S. 
Department  of  Energy  grant. 

We’re  now  working  with  the  Survey 
Research  Center  in  evaluating  five 
more  programs: 

•  The  followup,  previously  men¬ 
tioned,  to  the  solar  water  heating 
workshops  that  will  show  how  many 
participants  actually  installed  a  solar 
water  heating  system. 

•  A  survey  of  clients  who  were 
unable  to  attend  one  of  our  solar 
water  heating  workshops  but 
lequested  the  construction/installa¬ 
tion  manual  that  we  distribute  at  the 
workshops.  This  evaluation  will 
show  how  useful  the  manual  is 
alone. 

•  A  monthly  newsletter  on  energy 
topics  that  is  written  by  one  of  the 
energy  agents  and  mailed  to  about 
800  people.  From  this  we  hope  to 
learn  how  recipients  use  the  news¬ 
letter  and  if  it  helps  them  make 
energy-conserving  decisions. 

•  A  project  in  which  we  are  using 
the  multiplier  effect  by  working 
through  volunteers.  In  this  project, 
the  agent  arranged  with  the  county 
health  department  for  sanitarians  to 
suggest  and  demonstrate  low-flow 
shower  heads  to  hotels  and  motels 
when  the  sanitarians  make  their 
regular  twice-a-year  inspections. 

The  evaluation  will  be  designed  to 
show  how  much  energy  is  being 
saved  as  a  result  of  this  effort. 

•  A  followup  to  the  survey  on  the 
telephone  inquiries  we  receive.  As 
part  of  this  evaluation  we  will  obtain 
before-and-after  utility  records  of 
those  who  said  they  took  energy 
conservation  measures  as  a  result  of 
the  information  we  provided. 

The  utility  records  will  show  how 
much  energy  these  measures  save.  □ 
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Resident  Program  Aides  Make  a 

Gloria  J.  Boyd 

Extension  Communications  Specialist 
Prairie  View  A&M  University.  Texas 


During  the  past  eleven  years,  1890 
Extension  programs  have 
demonstrated  that  paraprofessionals 
are  extremely  effective  in  reaching 
non-traditional  clientele.  This  has 
indeed  been  the  case  for  the  1890 
Extension  program  in  Texas;  para¬ 
professionals  continue  to  be  the 
glue  of  its  one-on-one  concept. 

The  paraprofessionals,  also  referred 
to  as  program  aides,  working  with 
the  Cooperative  Extension  Program 
at  Prairie  View  A&M  University  in 
Texas,  serve  as  catalysts  in  helping 
non-traditional  clientele  become 
aware  of  and  adopt  new  self- 
improvement  practices. 

The  paraprofessionals'  effectiveness 
is  due  to  the  fact  that  they, 
themselves  are  selected  from  the 
clientele  group  they  intend  to  help. 
County  agents,  with  the  help  of 
program  specialists,  identify  a 
person  who  is  a  resident  of  the 
county  he  expects  to  serve.  They 
look  for  people  who  have  the  ability 
to  gain  acceptance  in  their  respec¬ 
tive  communities.  Paraprofessionals 
come  from  diverse  backgrounds 
such  as  ministers,  farmers,  and 
community  leaders. 

Supervised  By  County  Agents 

The  paraprofessionals  are  employed 
directly  by  the  1890  program,  but 
work  under  the  supervision  of  a 
designated  county  Extension  agent 
of  the  1862  Extension  Service.  The 
paraprofessionals  receive  training 
and  backup  support  for  Extension 
specialists  and  faculty  members 
from  both  Prairie  View  A&M 
University  and  the  Texas 
Agricultural  Extension  Service. 

Since  the  paraprofessionals  come 
from  similar  backgrounds  as  their 
clientele  they  can  often  readily 
identify  the  needs  of  this  group. 
Additional  areas  of  need  for  limited 
income  families  are  identified  by 
program  specialists,  county 
Extension  agents  and  of  course,  the 


target  audience  themselves.  The 
paraprofessionals  attend  training 
sessions  at  Prairie  View  A&M 
University  and  also  in  the  county 
and  district  offices  on  a  periodic 
basis.  These  sessions,  conducted  by 
program  specialists  in  coordination 
with  county  agents  keep  the 
program  aides  up-to-date  on 
subject  matter  to  transfer  to  their 
clientele.  Several  publications, 
specifically  designed  for  limited 
income  audiences,  are  shown  to  the 
program  aides  at  the  training 
sessions  so  they  can  share  them  with 
their  target  audience. 

Training 

Since  there  are  four  areas  of 
emphasis  in  the  1890  Extension 
Program  in  Texas — agriculture, 
home  economics,  community 
development,  and  4-H  and  youth — 
the  program  aides  receive  varying 
amounts  of  training  according  to 
the  demand  and  frequency  of 
change  in  each  program  area.  For 
example,  the  home  economics 
component  may  emphasize 
nutrition  one  month  and  the 
displaced  homemaker  the  next.  The 
4-H  and  youth  component  may 
have  a  continuous  effort  of  helping 
limited  income  youth  become 
productive  citizens. 

Once  the  program  aides  have  been 
selected  and  trained,  they  are  ready 


Difference 


to  employ  the  one-on-one  concept 
in  reaching  non-traditional 
clientele.  Paraprofessionals  make 
farm,  home,  or  community  visits  to 
their  clientele  on  a  weekly  basis. 

The  paraprofessionals  are  able  to 
talk  to  the  clientele  on  a  one-on- 
one  basis,  using  the  common 
language  or  jargon  that  is  familiar  to 
the  people.  This  helps  the  limited 
resource  individual  or  family  relax 
and  gain  confidence  in  the  para¬ 
professionals.  Once  this  confidence 
is  established,  helping  the  individual 
or  family  in  whatever  their  needs 
are  is  less  complicated. 


Accomplishments 

The  1890  Extension  paraprofes¬ 
sionals  in  Texas  have  been 
instrumental  in  helping  families  and 
individuals  weatherize  houses, 
install  water  systems  in  com¬ 
munities,  grow  their  own 
vegetables,  help  small  farm  pond 
owners  produce  and  market  over 
300,000  pounds  of  catfish,  obtain 
medical  assistance,  and  secure 
employment  for  youth. 

In  some  instances,  paraprofessionals 
work  with  other  agencies  and 
organizations  who  are  better  able  to 
handle  certain  target  audience 
concerns.  This  brings  about  a  good 
working  relationship  between 
Extension  and  other  agencies. 

Paraprofessionals  are  also 
instrumental  in  identifying 
volunteer  leaders  who  are  very 
important  to  the  success  of  the  1890 
Extension  Program  in  Texas.  Since 
the  program  has  begun,  volunteer 
leaders  have  secured  meeting  places, 
helped  organize  youth  clubs,  served 
as  translators,  and  assisted  in 
camping  projects. 

To  determine  the  effectiveness  of 
the  program  aides  and  their  impact 
on  limited  income  audiences,  the 
paraprofessionals  keep  an  individual 
file  on  each  person  in  their  clientele 
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Resident  program  aides  are  proving  to  be  ex¬ 
tremely  effective  reaching  non-traditional  clien¬ 
tele  in  the  1890  Extension  program  in  Texas. 
Whether  helping  small  livestock  producers, 
demonstrating  canning  techniques  to  youths,  or 
helping  families  grow  their  own  vegetables,  resi¬ 
dent  program  aides  are  proving  the  one-on-one 
concept  works. 


Monthly  Reports 

Monthly  contact  reports  are  also 
prepared  by  the  paraprofessionals 
which  indicate  the  number  of 
persons  they  contact  on  a  monthly 
basis  and  also  identifies  the  sex, 
racial  and  ethnic  groups  of  the 
clientele.  Monthly  narrative  reports 
are  done  by  the  program  aides  in 
which  detailed  information  is  given 
on  the  activities  they  have 
conducted  with  their  specific  target 
audiences  for  each  month. 


Not  only  have  the  paraprofessionals 
been  effective  in  assisting  limited 
income  audiences,  they  have  also 
benefited  themselves.  Several 
program  aides  have  furthered  their 
education  and  secured  Extension 
work.  Others  have  received 
recognition  for  their  work. 

Recently,  one  paraprofessional 
working  with  the  agricultural 
component  received  an  outstanding 
service  award  from  the  1890 
Extension  Program  in  Texas  for  his 
work  in  assisting  small  farmers. 
These  are  just  a  few  accomplish¬ 
ments  made  by  the  paraprofes¬ 
sionals. 


group.  This  helps  them  maintain  a 
record  on  their  clientele’s  progress. 
Paraprofessionals  also  make 
monthly  followup  visits  to  their 
clientele  to  check  on  their  progress. 


The  paraprofessionals  are  also 
expected  to  work  with  building  and 
advisory  committees  to  plan 
programs  for  limited  income 
clientele.  These  committees  help 
the  program  aides  keep  abreast  of 
educational  activities  to  be 
conducted  in  their  respective 
counties. 


The  paraprofessionals,  through  the 
use  of  the  one-on-one  concept  in 
assisting  limited  income  audiences, 
will  continue  to  live  up  to  the 
Extension  philosophy,  “Helping 
People  to  Help  Themselves.”  □ 
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Tennessee  Turns  To  TIS 

Cecil  E.  Carter,  fr. 

Professor  of  Agricultural  Extension 
Education 

University  of  Tennessee 


The  Tennessee  Information  System 
(TIS)  is  a  statewide  approach  used  in 
Tennessee  to  obtain,  analyze,  and 
use  survey  data  from  Extension 
clientele  and  Weekly  Activity 
Reports  (WAR)  required  from  staff 
as  part  of  the  Tennessee  Extension 
Management  Information  System 
(TEMIS).  Program  planners  use  TIS 
as  an  aid  in  program  planning, 
monitoring,  and  evaluating  impacts. 
TIS  provides  two  types  of  data: 
clientele  inputs  and  program  opera¬ 
tions — within  all  major  subject 
matter  areas.  Data  on  program 
outcomes  are  generated  by 
synthesizing  the  other  two  types  of 
data. 

TIS  attempts  to  provide  the  informa¬ 
tion  needed  to  carry  out  each  step 
in  the  program  development 
process  successfully.  The  idea  is  that 
optimal  execution  of  program  plan¬ 
ning,  implementation,  and 
evaluation  depends  at  least  partly 
on  the  quality  and  utility  of  informa¬ 
tion  from  TIS. 

Major  Components  of  TIS 

In  TIS,  clientele  information 
represents  the  environmental 
conditions  and  behaviors  of  a  target 
audience  at  a  given  point  in  time. 
They  reflect  the  learning,  growth, 
development,  and  potential  for 
improvement  of  this  target  group  in 
a  particular  planning  or  subject 
matter  area.  County  Extension 
agents  obtain  the  information 
through  personal  interviews  with 
persons  selected  at  random  within  a 
particular  target  audience. 

Data  on  Extension  educational  pro¬ 
gram  operations  in  Tennessee  come 
from  the  TEMIS  Weekly  Activity 
Reports.  They  help  answer  questions 
such  as  how  many  clients  there  are 
in  a  given  target  audience  and  what 
kind  of  services  they  receive. 

Program  outcomes  refer  to  progress 
toward  meeting  the  objectives  of 
the  educational  program.  Outcomes 
data  are  obtained  from  clientele 


inputs  data.  Traditional  descriptions 
used  in  TIS  have  been  changes  in 
conditions  and  behaviors  of  the 
group  (such  as  change  in  practices 
used,  change  in  yield  per  acre  or 
per  animal  unit). 

Developing  Programs 

Tennessee  Extension  staff  develop 
programs  through  (a)  identifying 
problems,  (b)  specifying  long-range 
(4-year)  objectives,  and  (c) 
designing  program  strategy  and 
actions  to  be  taken.  Eight  separate 
processes  can  be  identified  and 
discussed  briefly. 

First,  the  social  values  of  the  client 
group  and  the  legislative  mandates 
to  Extension  must  be  understood. 
Second,  Extension  staff  study 
environmental  conditions  and  the 
behavioral  situation  of  the  client 
group  to  see  to  what  degree  their 
desired  values  are  being  achieved 
and  Extension’s  legislative  mandates 
are  being  fulfilled.  Third,  in  needs 
assessment,  the  desired  conditions 
and  behaviors  for  the  client  group 
are  compared  with  existing 
conditions  and  behaviors.  The  gap 
between  desired  and  actual  condi¬ 
tions  and  behaviors  is  the  inferred 
need  or  problem. 

Based  on  inferred  needs  or 
problems,  the  Extension  staff 
identify  the  level  of  achievement 
(amount  of  change  in  conditions  or 
behaviors)  aimed  at  over  a  4-year 
period,  the  fourth  step  or  process. 

From  the  program  planning  portion 
of  TIS,  Extension  staff  move  into 
designing  and  implementing 
strategies  to  bring  about  the 
specified  changes  in  the  environ¬ 
ment  or  behaviors  of  the  client 
group — process  five.  Extension  staff 
work  closely  with  county  agriculture 
committees,  who  approve  all  new 
staff  and  programs,  and,  in  some 
counties,  work  also  with  study 
groups,  commodity  groups,  and 
other  lay  leadership.  In  the  sixth 
step,  to  learn  program  outcomes. 


Extension  staff  obtain  data  from 
each  of  the  other  data  base 
components  in  TIS — clientele  inputs 
and  program  operations. 

The  seventh  process  or  step,  is  to 
evaluate  impacts  of  programs,  and 
the  final  step  or  eighth  process 
involves  communication  and  use  of 
impact  evaluation  information 
internally. 

Not  a  New  Idea 

Basically,  this  description  of  TIS  and 
uses  of  TiS  is  an  effort  to  explain 
how  Tennessee  has  dovetailed 
routine  staff  reporting  (TEMIS)  with 
systematic  clientele  surveys  to 
produce  a  more  comprehensive 
approach  to  data  collection  than 
had  been  used  before.  These  data 
are  complementary  and  provide  the 
basis  for  analysis  and  evaluation  of 
the  program  development  process 
and  program  outcomes. 

Uses  of  TIS 

Program  evaluation,  as  viewed  in 
TIS,  includes  obtaining,  analyzing, 
interpreting,  feedback,  and 
application  of  information  about 
clientele  inputs,  program 
operations,  and  program  outcomes 
for  purposes  of  program  planning, 
program  monitoring,  and  program 
impact  evaluation.  Thus,  program 
evaluation  takes  place  throughout 
the  development  of  programs.  TIS 
provides  data  for  evaluation 
purposes  before,  during,  and  after 
programs  are  implemented,  on  a 
continuing  basis,  for  all  major 
subject  matter  areas. 

TIS  can  be  described  as  a  data  bank 
system.  Data  about  staff  inputs  into 
Extension  programs  are  available 
continually,  summarized  for 
program  planning,  analyzed  to  help 
monitor  programs,  and  used  to  look 
at  the  relationships  among  clientele 
inputs,  program  operations  and 
program  outcome.  The  result  is  a 
systems  approach  to  Extension  pro¬ 
gram  planning,  implementation, 
and  evaluation.  □ 
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Programing 

Tomorrow’s 

Tourists 


Michael  W.  Duttweiler 

New  York  Sea  Grant  Extension  Program  Coordinator 
and 

Michael  P.  Voiland 
Regional  Specialist 

New  York  Sea  Grant  Extension  Program 
Cornell  University,  Ithaca,  New  York 


Longer  range  planning  and  needs 
assessment  implicitly  involves  pro¬ 
jecting  the  future  from  current 
situations.  It  is  exceptionally  difficult 
to  draw  upon  past  experiences  and 
current  situation  descriptions  with¬ 
out  becoming  trapped  in  existing 
assumptions.  We  attempted  an 
experiment  in  Extension  program 
planning  which  included  distinct 
efforts  to  break  from  preconceived 
assumptions.  This  program  planning 
is  meant  as  a  supplement  to  formal 
needs  assessment — not  as  a 
substitute  for  it. 

The  Situation 

Tourism  development  is  viewed  as  a 
viable  economic  development 
component  by  many  Great  Lakes 
communities.  In  New  York,  the  Sea 
Grant  Extension  Program  of  Cornell 
Cooperative  Extension  and  the  New 
York  Sea  Grant  Institute  have 
assisted  lakeshore  communities  in 
dealing  with  the  boons  and  banes  of 
tourism  over  the  past  10  years. 
Primary  clientele  include  tourism 
businesses,  chambers  of  commerce, 
and  local  governments. 

By  early  1980,  Sea  Grant  and  other 
staff  with  responsibilities  in  recrea¬ 
tion  and  tourism  development  iden¬ 
tified  significant  changes  occurring 
in  New  York’s  coastal  tourism 
industry.  These  changes  were  the 
result  of  shifts  in  economic  climate 
and  dramatic  increases  in  tourism 
promotion  and  development. 
Careful  planning  on  the  part  of  Sea 
Grant  was  necessary  to  maintain 
high  impact  programs.  An  effective 
advisory  committee  system, 
statewide  situation  statements,  and 
strong  plans  of  work  were  in  hand 
but  we  were  uneasy  about  our 
preparedness. 

We  sought  a  way  of  looking  at 
ourselves,  our  capabilities,  and  our 
client’s  needs  through  the  window 
of  the  future. 


The  Approach  about  them.  A  healthy  climate  had 

Initial  efforts  were  important  but  been  established  for  our  primary 
hardly  innovative.  A  dual-purpose  project — developing  an  effective 

staff  workshop  was  conducted  to  plan  of  action  for  tourism 
review  organizational  strategies,  programing, 

structure,  and  emphases,  and  to 

initiate  planning  specifically  for  The  approach  derived  for  planning 

recreation  and  tourism.  Sea  Grant  community  development  efforts 
and  Cornell  Community  Resource  related  to  tourism  took  a  decep- 
Development  staff  participated — a  tively  simple  form.  The  following 
total  of  15  persons.  items  became  our  primary  planning 

tools: 

At  our  first  workshop  we  critiqued 

how  well  we  were  delivering  pro-  •  Review  and  critique  of  past 
grams.  General  suggestions  for  efforts; 

improvement  followed — such  as  •  Organizational  improvement 

need  for  more  effecive  use  of  media  objectives  identified  through 
and  audience  multipliers  and  for  workshops; 

increased  emphasis  on  leadership  •  “Futurescapes”  for  1985. 
development.  In  a  followup 

meeting  Sea  Grant  staff  translated  Analyzing  these  items  together 

these  into  a  collective  “contract”,  should  add  up  to  our  revised  situa- 

which  specified  methodological  tion  statement  and  goals  for  the  3  to 

improvements  to  be  incorporated  in  5  years. 

individual  plans  of  work.  These 

commitments  were  reinforced  Critique 

through  regular  review  and  The  critique  of  past  efforts  included 

discussion  during  performance  compilation  and  analysis  of  all  plans 

review.  of  work  dealing  with  recreation  and 

tourism,  review  of  annual  reports. 

While  not  solely  related  to  recrea-  review  of  more  than  25  existing 
tion  and  tourism,  these  organization  program  evaluations,  and  compila- 
improvement  objectives  played  a  tion  of  program  resources  such  as 
key  role  in  helping  to  identify  short-  research  reports  and  teaching 

comings  in  our  programs  and  make  materials  produced  over  the  past 

commitments  to  do  something 
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frequent  and  more  selective  recrea¬ 
tional  travel.  The  transportation 
topic  contained  projections  for 
personal  auto  use  and  the 
development  of  more  public  trans¬ 
portation.  Other  topics  included 
communication  (supply  of  visitor 
information  and  availability  of  com¬ 
puterized  travel  information)  and 
destination  areas  (the  formation  of 
tourism  associations  and  the  group 
packaging  of  tourist  attractions). 

The  futurescapes  then  were 
synopsized  through  group 
discussion  that  focused  on  primary 
trends  likely  to  influence  our  key 
clientele.  The  tourist  trends 
predicted  for  the  Lake  Ontario  area 
were  changes  in  recreational  and 
tourism  leisure  needs  such  as  use  of 
smaller  boats,  longer  stays  within  an 
area,  and  increased  emphasis  on 
physical  fitness  exercises.  Critical 
importance  was  noted  for  tourism 
packaging  and  coordinated  pro¬ 
motion.  And  water  use  conflicts 
between  boaters  and  anglers  and 
between  sport  and  commercial 
fishermen  were  projected  as 
increasing. 

Major  Themes 

The  summary  of  futurescapes  was 
used  to  reformulate  our  longer 
range  goals  for  recreation  and 
tourism  related  programs.  The 


seven  years.  Each  aspect  was  sum¬ 
marized  in  a  1  to  3  page  discussion 
document.  These  sources  distilled 
into  a  general  sense  of  major  pro¬ 
gram  themes,  successes  and  failures 
in  each,  program  trends  and  present 
capabilities  clarified  through 
extensive  discussion. 


“Futurescapes”  for  1985 

Futurescapes,  the  only  new  item  in 
the  formula,  were  meant  to  be 
“mind  openers”  to  help  anticipate 
programing  circumstances  in  the 
future.  The  five  specialists  with 
tourism-related  responsibilities  inde¬ 
pendently  developed  written 
projections  of  the  1985  climate  for 
tourism  along  New  York's  Great 
Lakes.  General  topics  assigned 
included  tourist  businesses,  recrea¬ 
tional  participation,  access  devel¬ 
opment,  and  environmental  quality. 
Each  used  a  variety  of  predictive 
documents — such  as  travel  and 
tourism  trends — and  a  healthy  dose 
of  professional  judgment.  We  pur¬ 
posely  did  not  prescribe  a  uniform 
format — to  avoid  prejudicing  the 
writers. 


mechanism  was  a  third  and  final 
staff  work  session.  Major  themes 
that  resulted  were  as  follows: 


•  Knowledge  about  Lake  Ontario 
recreational  opportunities  and 
associated  technical  and  safety  con 
siderations  typically  is  limited.  For 
coastal  tourism  to  be  a  true 
economic  stimulus,  such  informa¬ 
tion  must  be  readily  available  to  a 
very  broad  population. 


Program  Implication:  Increased  use 
of  media  and  multiplier 
organizations  for  distribution  of 
recreation-related  information. 


Staff  investigated  the  regional 
tourism  market  in  1985  and  how  it 
would  be  affected  by  such 
projections  as  the  decline  in 
regional  population,  and  less 


•  The  economic  role  of  tourism  and 
the  steps  required  for  proper 
tourism  management  are  poorly 
recognized  by  many  coastal 
communities. 
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The  New  York  Sea  Grant  Extension  program  has 
formulated  long-range  planning  for  tourist  de¬ 
velopment  in  that  state's  Great  lakes.  "Future- 
scapes"  are  the  tourist  specialists'  way  of  an¬ 
ticipating  programing  circumstances  in  the 
future. 


Program  Implication:  Tourism 
development  programs  must 
include  a  component  aimed  at 
clearly  demonstrating  economic 
impacts  and  management  options. 

•  Efforts  to  expand  use  of  coastal 
recreation  resources  while  protect¬ 
ing  their  quality  will  require  a  broad 
constituency  on  their  behalf. 

Program  Implication:  Past  educa¬ 
tional  programs  that  had  em¬ 
phasized  angling  interests  must  be 
expanded  to  a  broader  range  of 
user  groups. 

•  Existing  marine  trades  facilities 
are  inadequate  to  handle  expected 
growth  in  need  for  recreational 
services. 

Program  Implication:  Current 
efforts  to  assist  marine  trade 


businesses  should  be  intensified, 
especially  in  the  areas  of  business 
management  and  facilities  design. 

Specific  organizational  objectives 
and  performance  indicators  were 
written  for  each  theme  by  the 
program  coordinator.  These  have 
served  as  primary  tools  for 
individual  plan  of  work  formulation 
and  suggesting  future  research 
directions. 

The  Real  Products 

The  planning  documents  generated 
were  far  from  static.  In  the  months 
after  a  “final”  draft  was  prepared, 
significant  modifications  were 
made.  Our  detailed  situation  state¬ 
ment  and  program  plan  was  revised 
and  program  strategies  continue 
evolving. 

In  some  ways,  the  key  products  of 
the  process  were  outside  the  actual 
documents.  Most  importantly, 
everyone  with  major  responsibility 
in  our  Great  Lakes  recreation  and 
tourism  programming  had  a  direct 
and  significant  role  in  the  review 
and  planning  process.  Gains 
because  of  this  corporate  approach 
were; 

•  Discovery  of  a  collective  sense  of 


purpose  and  direction  in  what  had 
been  a  somewhat  diffuse  program 
unit. 

•  An  increased  willingness  to  shift 
from  past  program  practices  in 
anticipation  of  trends  collectively 
identified. 

•  An  increased  ability  to  alert  and 
counsel  researchers  about  antic¬ 
ipated  information  needs. 

•  Creation  of  a  broad  framework 
for  counseling  staff  in  program 
development. 

A  New  Consensus 
An  internal  program  planning 
procedure  was  described  which 
utilized  review  of  plans  of  work, 
annual  reports,  and  program 
evaluations;  organizational  improve¬ 
ment  objectives;  and,  futurescapes 
of  program  content  five  years 
hence.  Needed  program  improve¬ 
ments  were  identified  and  resulted 
in  extensive  modification  of  plans  of 
work  and  delivery  strategy.  The 
general  procedures  followed  appear 
to  be  appropriate  for  a  broad  range 
of  community  development 
program  planning  situations. 

Perhaps  the  most  significant 
outcome  was  staff  consensus  on 
program  purposes  and  direction 
and  increased  willingness  to 
critically  evaluate  efforts. 

More  detailed  descriptions  of  pro¬ 
cedures  used  and  copies  of  key 
documents  generated  can  be 
obtained  from  the  authors  as  can 
copies  of  a  formal  survey  of  tourism 
businesses  used  to  provide 
“outside”  validation  of  trends 
identified  by  staff.  This  later  project 
involving  interviews  with  nearly  100 
tourism  industry  leaders  largely  con¬ 
firmed  the  “findings”  of  our 
internally  generated  futurescapes.  It 
appears  that  major  perceived  needs 
of  audiences  can  be  effectively 
addressed  through  careful  attempts 
to  avoid  dominance  of  current 
program  assumptions.  □ 
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Sampling  The  Best 

Michael  Q.  Patton 
Program  Director 

Caribbean  Agricultural  Extension  Project 
University  of  Minnesota,  5t.  Paul 


Evaluating  the  Caribbean  Agricul¬ 
tural  Extension  Project  presents  an 
interesting  problem.  The  solution  to 
that  problem  involves  a  purposeful 
sampling  strategy  that  may  be 
applicable  in  other  Extension  evalua¬ 
tion  situations. 

The  problem:  how  to  provide  the 
United  States  Agency  for  Interna¬ 
tional  Development  (AID)  with 
information  on  the  long-term 
potential  impact  of  Extension  in  the 
Caribbean. 

It  is  important  to  understand  the 
situational  context  within  which  this 
issue  is  being  addressed.  In  1980, 

AID  funded  the  Caribbean  Agricul¬ 
tural  Extension  Project.  The  project 
is  a  joint  endeavor  between  the 
University  of  the  West  Indies  and 
the  Midwest  Universities  Con¬ 
sortium  for  International  Activities 
(MUCIA)  with  the  University  of  Min¬ 
nesota  as  the  lead  university. 

During  phase  I,  from  1980  to  1982, 
project  staff  worked  with  Ministry  of 
Agriculture  officials  in  eight  Carib¬ 
bean  countries  to  evaluate  the 
effectiveness  of  their  Extension 
services  and  develop  plans  for 
national  Extension  improvement. 
The  countries  involved  were 
Antigua,  Belize,  Dominica,  Grenada, 
Montserrat,  St.  Kitts/Nevis,  St.  Lucia, 
and  St.  Vincent.  In  1982,  AID 
funded  phase  II  which  is  aimed  at 
implementation  of  the  National 
Extension  Improvement  Plans 
developed  in  phase  I. 

Proving  Cost-Effectiveness 

Shortly  after  the  signing  of  the 
phase  II  contract  a  conference  was 


held  for  directors  of  AID  missions  in 
Latin  America  and  the  Caribbean. 
The  dominant  view  at  that  meeting, 
as  reported  to  project  staff,  was  that 
funding  Extension  projects  in 
developing  countries  was  not  an 
effective  use  of  AID  funds.  It  is  in 
the  context  of  this  skepticism  about 
the  cost-effectiveness  of  agricultural 
Extension  projects  in  developing 
countries  that  the  Caribbean  project 
was  asked  to  provide  data  on  the 
long-term  potential  of  agricultural 
Extension  in  the  Caribbean. 

The  immediate  problem  is  that  the 
Caribbean  Agricultural  Extension 
Project  is  an  institution  building 
effort  aimed  at  improving  national 
Extension  services  through  training, 
consultation,  staff  development,  and 
program  planning.  During  the 
current  life  of  the  project  from  1982 
to  1985,  it  is  fair  to  expect  major 
changes  in  the  organization  and 
delivery  of  agricultural  Extension, 
and  such  changes  can  be  evaluated. 
However,  impacts  on  farmers  will  be 
marginal  during  this  period.  Many 
of  the  Extension  staff  will  not  return 
from  training  until  late  in  the  life  of 
the  project.  Because  changes  in 
organization  are  timed  to  occur 
gradually  throughout  the  life  of  the 
project,  it  will  take  some  time  for 
measurable  changes  in  Extension 
organization  and  delivery  systems  to 
show  up  in  changed  farmer 
practices. 

How,  then,  can  the  question  of 
long-term  Extension  effectiveness  in 
the  Caribbean  be  addressed?  At  this 
point  no  data  exist  on  the 
parameters  of  potential  Extension 
effectiveness.  That  is,  there  are  no 
reliable  data  on  Extension  impact 
from  which  future  projections  of 
effectiveness  can  be  made.  How¬ 


ever,  a  “purposeful”  sampling 
strategy  within  the  current  life  of 
the  project  makes  it  possible  to 
provide  data  that  can  be  used  to 
establish  the  needed  parameters  of 
effectiveness.  Such  data  can  help 
answer  the  question  of  what  can  be 
expected  from  Extension 
development  over  time. 

Measuring  Maximum  Impact 

Purposeful  sampling  is  a  qualitative 
methods  approach  aimed  at 
carefully  selecting  for  study  a  small 
number  of  cases  that  will  yield  a 
great  deal  of  information  for  policy¬ 
making  purposes.  The  strategy 
adopted  in  this  case  is  aimed  at 
measuring  the  maximum  impact 
that  Extension  might  have  under 
fairly  ideal  conditions. 

The  sampling  strategy  begins  with 
identification  in  each  participating 
country  of  an  “outstanding  agricul¬ 
tural  Extension  agent.”  The  outstand¬ 
ing  agent  is  identified  through  a 
nomination  and  selection  process 
directed  by  the  National  Planning 
Committee  for  Extension 
established  in  each  country  during 
phase  I  of  the  project.  These 
committees  include  farmers, 
representatives  of  farmer  organiza¬ 
tions,  Ministry  of  Agriculture 
officials,  agricultural  researchers, 
and  others  with  interests  in  agricul¬ 
tural  development.  Following 
selection  of  outstanding  agents, 
they  are  asked  to  identify  the  five 
farm  families  with  which  they  have 
worked  most  closely  and  on  which 
they  have  had  the  greatest  impact. 
Case  studies  will  then  be  made  of 
each  of  these  families  to  establish 
the  nature  and  extent  of  the 
Extension  impact. 
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Sampling  Strategy 

This  sampling  strategy  will  yield 
forty  cases  where  agricultural  Exten¬ 
sion  has  had  an  important  impact  in 
the  Caribbean.  This  sample  is 
purposefully  biased  to  establish 
maximum  likely  impact,  or  to 
establish  the  outside  ideal 
parameters  for  Extension  effective¬ 
ness.  It  can  be  expected  that  the 
typical  Extension  agent  will  have 
somewhat  less  impact  than 
outstanding  agents.  However,  by 
gathering  data  about  the  ideal  that 
might  be  obtained,  it  is  possible  to 
establish  the  maximum  that  could 
be  expected  as  an  increasing 
number  of  agents  begin  to  operate 
in  the  area. 

In  effect,  this  evaluation  will  provide 
AID  with  information  about  the 
maximum  potential  long-term 
impact  of  agricultural  Extension  pro¬ 
grams  on  farm  families  in  the 
Caribbean.  The  data  will  establish  an 
ideal  for  Extension  Service  effective¬ 
ness.  That  ideal,  and  projections 
downward  from  that  ideal,  can  then 
be  used  in  policy  discussions  to 
decide  if  the  level  of  possible 
impact  is  worth  attempting  to  attain 
through  projects  that  improve 
agricultural  Extension  services. 

This  is  an  evaluation  sampling 
strategy  where  bias  is  purposefully 
built  in  at  every  stage  to  increase 
the  usefulness  of  the  information 
obtained.  The  purposeful  bias  in  this 
kind  of  sampling  becomes  a 
strength  rather  than  a  weakness 
because  it  provides  important 
information  for  the  policy  process 
that  would  otherwise  be  unavail¬ 
able. 


Data  for  Program  Development 

The  evaluation  will  also  provide 
indepth  information  about  the  prac¬ 
tices  and  methods  of  outstanding 
Caribbean  agents.  This  data  will  be 
used  in  training  and  in  efforts  aimed 
at  promoting  increased  profes¬ 
sionalism  and  pride  among  Carib¬ 
bean  Extension  personnel.  Thus, 
detailed  case  study  descriptions  of 
model  agents  and  their  Extension 
work  with  farm  families  will  provide 
data  not  only  for  evaluation  but  also 
for  future  program  development 
and  training  activities.  □ 


To  evaluate  the  Caribbean  Extension  Project  a 
"purposeful"  sampling  strategy  began  by  identi¬ 
fying  an  outstanding  agricultural  Extension  agent 
in  each  of  eight  Caribbean  countries.  Here, 
Thomas  H.  Henderson,  (right),  project  director, 
and  director  of  Extension  at  the  University  of 
West  Indies,  congratulates  Anthony  Philgence  of 
St.  Lucia  for  being  selected  the  outstanding  Ex¬ 
tension  agent  for  the  Extension  Service  of  that 
country. 
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which  Comes  First — 
Problems  Or  Programs? 


Kathleen  Beery 
Assistant  State  Leader.  CES 
Home  Economics  Program 
Iowa  State  University,  Ames 


In  Iowa,  home  economics  advisory 
committees  provide  a  rich  source  of 
information  on  people’s  needs  and 
problems  to  use  in  planning 
Extension  programs. 

One  role  of  the  economist  is  to 
involve  advisory  committee  mem¬ 
bers  in  the  program  planning 
process.  How  do  you  assist  a  com¬ 
mittee  in  identifying  families’ 
needs? 

Needs  Assessment  Study 

Ideally,  committee  members  are 
involved  in  fact-finding  and  advisory 
roles.  A  formalized  needs  assess¬ 
ment  process  could  help  staff  and 
committee  attain  concrete  informa¬ 
tion  for  program  planning. 

Linda  Swenson,  Extension  home 
economist,  conducted  a  study  as 
part  of  her  master’s  program  in 
which  volunteer  advisory  committee 
members  assessed  needs  through  a 
card  sort  technique.  Swenson,  in 
conjunction  with  committee 
members  and  Extension  paraprofes- 
sionals,  generated  30  problem 
statements  in  family  living  and  food 
and  nutrition.  Problem  statements 
were  printed  on  small  cards. 
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Swenson  trained  advisory 
committee  members  to  interview 
clientele  by  use  of  the  card  sort 
technique.  Each  respondent 
selected  his  or  her  five  major  con¬ 
cerns  from  among  the  30  cards  and 
ranked  these  from  the  “greatest”  to 
the  “smallest”  problem.  Problems  of 
greatest  concern  to  clientele  were 
used  by  Swenson  and  the 
committee  to  develop  the  home 
economics  program. 

The  needs  assessment  became  an 
action-oriented  process.  Committee 
members  not  only  collected  data 
upon  which  to  build  their  program, 
but  they  also  had  direct  contact  with 
clientele  in  the  county.  Thus,  the 
support  base  for  Extension  was 
extended.  Clientele  also  had 
concrete  statements  to  rank.  The 
focus  of  this  process  was  on 
problems,  not  program<^.  Programs 
became  the  end  result  of  the  needs 
assessment. 

Another  positive  outcome  was  that 
advisory  committee  members  felt 
ownership  of  the  programs  which 
evolved  out  of  the  needs 
assessment. 

Needs  Assessment  Across  The  State 

The  process  of  using  “problem 
statements”  was  expanded  and 
adapted  for  use  across  the  state. 
Alternatives  for  using  this  approach 
were  developed  and  shared  with 
home  economists.  Some  of  the 
alternatives  reduced  the  time  and 
travel  commitment  required  of  the 
committees  and  staff. 

Additional  problem  statements  were 
written  to  supplement  those  in 
Swenson’s  study.  Those  statements 
and  the  following  suggestions  were 
shared  with  home  economists: 

•  Committee  members  could 
conduct  a  card  sort  interview  with 
clientele. 

•  Home  economists  could  conduct 
a  survey  in  the  monthly  newsletter. 


•  Committee  members  could 
conduct  a  personal  or  telephone 
interview  using  a  condensed  version 
of  the  problem  statements. 

•  Home  economists  could  send  a 
mailed  survey  to  a  random  sample. 

The  county  home  economists 
adapted  the  “problem  statement” 
approach  to  meet  their  needs.  Some 
home  economists  modified  the 
problem  statements  provided  and 
others  generated  their  own,  based 
on  problems  expressed  by  the 
advisory  committees.  The  advisory 
committees  conducted  card  sorts  or 
personal  interviews. 

An  Area  Works  On  The  Card  Sort 

Home  economists  and  advisory  com¬ 
mittees  in  the  Cedar  Rapids  area 
developed  a  set  of  problem  state¬ 
ments.  To  increase  the  awareness 
about  other  families’  problems,  the 
home  economists  presented  data 
and  trends  concerning  families  in 
Iowa  and  the  Nation.  Committees 
brainstormed  on  problems  in  their 
counties.  The  home  economics  staff 
wrote  a  set  of  statements  reflecting 
these  problems. 

The  card  sort  technique  was  used 
with  several  groups:  home 
economics  advisory  committees. 
Extension  councils,  and  other 
community  groups.  Ann  Harrison, 
Extension  home  economist, 
explains,  “It  is  a  less  abstract 
process,  so  clientele  could  more 
readily  give  their  opinions  as  to  the 
greatest  problems  facing  families.” 

Committees  and  the  Questionnaire 

Several  committees  helped  develop 
questionnaires  based  on  problem 
statements  and  tabulated  the  results. 
County  home  economics  news¬ 
letters  were  also  used  to  disseminate 
the  questionnaire. 

Sue  McDonnell,  Extension  home 
economist,  and  her  home 


economics  advisory  committee 
thought  the  questionnaire  was 
useful.  “The  committee  felt  better 
about  the  programs  for  the  year. 
They  felt  as  if  they  were  on  firmer 
ground  in  the  planning  process,” 
she  says.  “Programs  which  were 
developed  upon  problems  strongly 
emphasized  by  clientele  have  had 
very  successful  results.” 

Weight  control  was  identified  as  a 
priority  problem.  Attendance  was 
high  at  programs  developed  for  this 
problem  and  quite  successful.  “The 
committee  would  have  predicted 
this.  But  preretirement  planning 
also  came  out  strong,”  McDonnell 
recalls.  “We’ve  had  125  participants 
in  the  past  2  years,  a  response  we 
would  not  have  predicted.  We 
might  not  have  developed  these 
sessions  if  it  hadn’t  been  for  the 
survey.” 

Positive  Results 

Committees  and  clientele  were 
actively  involved  in  the  needs 
assessment  phase.  Committees  also 
helped  the  programs  and,  often, 
they  saw  the  benefits  of  building  on 
sound  information  about  families. 

Implications  For  Future  Planning 

The  home  economists  used  various 
techniques,  but  some  common 
threads  increased  the  effectiveness 
of  the  planning  process: 

•  Committees  were  involved  in  fact 
finding  and  analyzing  the 
information. 

•  Committees  helped  develop  the 
problem  statements. 

•  Problems  were  expressed  in  a 
concrete  form  so  that  clientele 
could  react. 

Most  important,  the  emphasis  was 
on  identifying  problems  first  and 
then  developing  programs.  □ 
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asked  her  to  dance  and  then  he  got 
her  to  dance  with  a  camper.  The 
next  day  her  cabin  won  a  volleyball 
game  and  she  was  a  major  con¬ 
tributor  to  the  success. 

How  measure  success? 

We  can  measure  the  number  of  4-H 
members,  the  number  of  4-H  volun¬ 
teers,  the  quality  of  the  4-H 
projects,  the  number  of  members 
and  volunteers  who  return  each 
year,  the  number  of  campers,  and 
the  amount  of  savings  to  everyone 
because  of  volunteers  in  4-H,  but 
can  we  measure  the  personal 
growth  and  long-term  effect  on 
attitudes  and  values  on  4-Hers  and 
campers? 

There  is  no  way  to  show  on  paper 
the  number  of  lives  touched  each 
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After  I  completed  a  week  with 
nearly  100  teenagers  at  a  nine 
county  leadership  development 
camp,  I  was  asked — “Was  it  a 
success?”  This  is  not  easy  to  answer. 
Did  everyone  have  fun?  Yes,  but  is 
that  a  measure  of  success?  Was 
there  a  major  impact  on  their  lives? 
Yes,  I  think  so.  What  did  the  evalua¬ 
tions  say?  They  said  the  participants 
felt  it  was  worthwhile  and  they  want 
a  longer  camp.  But  a  question  still 


How  much  long-term  impact  do  we 
^  really  have  through  4-H  activities 
and  particularly  4-H  camping? 


A  young  girl  attended  camp  1  year. 
She  was  not  very  outgoing,  nor  did 
she  really  want  to  be  at  camp.  She 
seemed  to  be  sitting  most  of  the 
time.  Then  one  evening  a  counselor 


This  is  not  an  article  about 
evaluation,  statistical  analysis,  or 
research;  it  is  one  about  people — 
young  people — whose  lives  and 
futures  are  what  4-H  is  all  about. 


The  Limits  of  Evaluation 


evaluations,  find  the  statistical  rela¬ 
tionships  between  dollars  spent  and 
numbers  of  members — all  of  which 
are  necessary.  But  how  do  we  know 
what  influence  a  counselor  had  on  a 
camper’s  life?  We  need  to  show 
numbers  and  measure  many  areas. 
But  did  we  touch  just  one  life  to 
help  a  person  see  something  about 
themselves  that  wasn’t  noticed 
before? 


We  in  Extension  are  really  being 
urged  to  measure  success,  use 


lingers. .  .  ."How  do  you  measure 
success  in  situations  where  attitudes 
and  long  term  value  clarification  is  a 
goal?”  How  do  we  measure  that 
counselor  who  was  a  camper  10 
years  ago  and  always  caused 
trouble,  but  is  now  a  school  teacher 
and  participated  in  this  week’s  camp 
as  a  counselor?  Did  we  help  him 
turn  his  life  around? 


William  C.  Smith 
4-H  Extension  Agent 
Canfield  Area  Extension  Center 
The  Ohio  State  University 


At  another  leadership  camp  many 
years  ago,  a  young  teenager  was 
caught  with  alcohol.  I  talked  with 
him  and  asked  what  he  thought  I 
should  do.  He  said  he  should  get  a 
second  chance.  Yes,  he  should  have 
been  sent  home  according  to  all  the 
rules,  but  he  wasn't.  He  came  back 
the  next  year  and  the  following  year 
he  asked  to  be  a  counselor  for  me  at 
junior  camp.  With  some  reserva¬ 
tions  he  was  selected.  He  became 
one  of  the  very  best  counselors  I 
have  ever  worked  with.  He  was  a 
big,  burly  character  whom  the  kids 
really  looked  up  to  and  became  the 
“teddy  bear”  of  camp. .  .  .being 


year  by  4-H  volunteers,  camp 
counselors,  or  professionals,  but  I 
can  say  it  does  happen.  1  have  no 
statistics;  only  stories  of  young 
people  becoming  quality  adults.  NO; 
I  can't  say  they  wouldn't  have 
become  what  they  are  without  4-H. 
But  I  can  say  we  were  there  to  help 
and  maybe  that  made  a  difference. 


Was  our  senior  leadership  camp  a 
success?  Absolutely!  We  touched  90 
young  people  and  gave  them  new 
experiences  and  new  relationships 
and  provided  leadership  oppor¬ 
tunities  for  them.  Maybe  10  years 
from  now  they  can  tell  me  how 
successful  we  were.  As  of  now,  I 
measure  success  by  the  smiles,  the 
tears,  and  the  future  families  and 
friendships. 

4-H  is  a  growth  experience  for  all 
those  involved.  It  is  people — young 
and  old — growing  and  learning.  If 
this  growth  is  accomplished  then  we 
have  been  successful.  □ 


hugged  and  admired  by  the  young 
campers. 

Was  that  leadership  camp  a  success 
so  many  years  ago?  Yes,  if  that  one 
person  was  given  a  second  chance 
and  grew  and  learned  from  the 
experience. 

Touching  Lives 

We  get  so  involved  in  4-H  programs 
or  camp  programs;  but  we  forget 
that  the  major  measure  of  success  is 
how  well  our  people  work  with  and 
guide  young  people.  Programs 
come  and  go,  campers  forget  that 
tournament  or  that  camp  carnival, 
but  they  seldom  forget  that 
counselor  who  got  them  up  and 
involved  when  they  were  feeling  so 
alone. 


extension  review /winter  1 984 


4-H  Priority  Programs 


Anne  L  Heinsohn 
Youth  extension  Specialist 
The  Pennsylvania  State  University, 
University  Park 


In  1981,  the  Pennsylvania  Coopera¬ 
tive  Extension  Service  launched  a 
program  planning  effort  designed  to 
direct  resources  to  high  priority 
programs. 

These  priority  programs  were 
identified  by  the  state  program 
committees  for  Agriculture,  4-H, 
Family  Living,  and  Community 
Resource  Development.  Extension 
agents  and  specialists  formed  task 
force  committees  in  each  area. 

These  task  forces  then  developed 
objectives,  content,  delivery,  and 
evaluation  methods — methods  that 
were  shared  with  Extension  agents 
during  a  statewide  program 
planning  session. 

These  plans  became  the  basis  for 
county  and  state  plans  of  work  for 
the  next  two  years. 

Priority  Programs 

State  4-H  youth  specialists  view 
these  priority  programs  as  an  oppor¬ 
tunity  to  increase  and  improve  pro¬ 
gram  evaluation  in  county  4-H  pro¬ 
grams;  the  priority  programs 
require  systematic  evaluation  to 
report  program  accomplishment.  As 
similar  programs  were  to  be 
conducted  throughout  the  state, 
evaluation  assistance  and  training 
could  focus  on  the  four  4-H  priority 
programs. 

Over  the  years  the  Pennsylvania 
Extension  Information  Management 
System  has  required  program  evalua¬ 
tion  in  the  county  plans  of  work  in 
order  to  report  program  accomplish¬ 
ment  in  end  of  year  narrative 
reports.  However,  while  individual 
county  programs  had  been 
reporting  program  success  stories,  a 
more  systematic  approach  to  pro¬ 
gram  evaluation  was  needed  to 
monitor  the  priority  programs. 

A  process  for  incorporating  sys¬ 
tematic  collection  of  evaluation  data 
into  the  entire  “life”  of  a  program, 
as  well  as  the  program  planning,  was 
needed.  In  this  way,  data  would  be 


in  hand  to  include  in  the  narrative 
reports.  In  short,  concrete  plans  for 
program  evaluation  needed  to  be 
developed  along  with  plans  to  carry 
out  a  program  so  data  collection 
could  be  accomplished  as  a 
program  proceeded.  An  increased 
emphasis  on  program  evaluation 
had  to  fit  into  and  be  an  integral 
part  of  regular  programing  rather 
than  an  additional  step.  A  point  for 
planning  and  carrying  out 
evaluation  needed  to  be  inserted 
into  the  already-in-place  program 
planning  implementation  and 
reporting  system.  Constance 
McKenna’s  article  “Making  Evalua¬ 
tion  Manageable”,  provided  the 
means  to  do  this. 

Tool  for  Evaluation 

McKenna  not  only  recommended 
the  Provus  Discrepancy  Evaluation 
Model  as  a  tool  for  evaluating  Exten¬ 
sion  programs,  but  she  translated 
the  model  into  language  familiar  to 
Extension  workers — language  that 
could  be  applied  to  the 
Pennsylv.ania  EMIS  system.  Since  the 
McKenna-Provus  model  is  designed 
to  be  applied  to  programs  after  they 
have  been  planned;  either  just  prior 
to  implementation  or  during  the 
program,  its  structure  could  be  used 
to  insert  systematic  evaluation  into 
prepared  county  plans  of  work  early 
in  the  program  year.  After  inter¬ 
preting  the  McKenna  model  to  re¬ 
ject  the  Pennsylvania  EMIS  plan  of 
work/task  format,  it  was  ready  to  be 
applied  to  “real”  county  plans  of 
work. 

First  Session 

A  presentation  was  developed  for 
use  with  county  4-H  staff.  The  first 
such  session  was  with  4-H  agents  in 
one  region  of  the  state.  The  region 
had  selected  the  4-H  Adult  Leader¬ 
ship  Priority  Program  for  emphasis 
during  the  program  year.  One  of 
the  tasks  in  leadership  development 
was  used  as  an  example  and  the 
model  was  applied  to  it.  In  addition 
a  handout  sheet  containing  sug¬ 
gested  evaluation  methods  for  the 
particular  task  was  presented. 


A  session  similar  to  the  one 
described  was  included  in  a  state¬ 
wide  4-H  Evaluation  workshop  and 
on  several  other  occasions  early  in 
the  1981-82  program  year.  Com¬ 
ments  from  workshop  participants 
were  quite  positive.  Several 
indicated  using  the  McKenna- 
Provus  model  with  the  usual  plan  of 
work  process  helped  them  see 
program  evaluation  as  part  of  the 
process,  rather  than  as  a  somewhat 
mysterious  extra  step.  Overall,  the 
workshop  participants  found  that 
the  model  helped  them  feel  more 
comfortable  with  the  whole  notion 
of  program  evaluation. 

Clear  Objectives 

The  most  immediate  result  was  a 
noticeable  improvement  in  both 
county  plans  of  work  (tasks)  and 
narrative  reports.  When  the  means 
to  accomplish  program  evaluation 
are  planned  and  prepared  for  at  the 
time  the  plan  of  work  is  being 
written  more  attention  is  given  to 
writing  clear,  measurable  objectives. 
Vague  objectives  cannot  be  tole¬ 
rated  when  evaluation  is  to  take 
place.  This  became  quite  clear  and 
was  recognized  by  agents  exposed 
to  the  model. 

County  narrative  reports  submitted 
at  the  end  of  the  program  year  con¬ 
tained  the  results  of  more  systematic 
evaluation.  Program  accomplish¬ 
ment  was  supported  by  evidence  of 
clientele  reaction  and,  in  some 
cases,  knowledge  gain.  This  con¬ 
stituted  a  marked  improvement 
over  narrative  reports  that  simply 
indicated  a  program  did  take  place. 

Finally,  for  those  agents  exposed  to 
the  model,  it  not  only  has  helped 
the  individuals  develop  more  posi¬ 
tive  attitudes  to  program  evaluation, 
but  has  contributed  to  a  more 
positive  overall  climate  for  evalua¬ 
tion.  Such  a  climate  will  do  much  to 
further  program  evaluation  in 
Pennsylvania.  □ 
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Documenting  Indicators 

Ruth  M.  Conone 

Slate  Program  Coordinator  for  Human  Resource  Development.  CES 
Montana  State  University,  Bozeman 


One  of  the  great  cfiallenges  in 
evaluation  of  Extension  programs  is 
to  document  social  indicators  of 
program  effect. 

Since  many  variables  affect  how  an 
individual  experiences  a  program,  it 
is  usually  difficult  to  identify  social 
cause  and  effect  relationships. 
However,  correlational  relationships 
can  be  documented. 

In  order  to  correlate  indicators  of 
intended  effect  with  program  input, 
desired  learner  behavior  must  be 
clearly  identified  in  the  program 
objective.  Indicators  of  the  desired 
learner  behavior  need  to  be  defined 
in  measurable  terms.  If  the 
indicators  are  drawn  from  con¬ 
ceptual  frameworks,  the  data 
gathered  in  assessment  can  con¬ 
tribute  to  theory  construction  and 
thereby  strengthen  our  ability  to 
explain  social  phenomena. 


County  agent  roles  are  to  identify 
desired  change,  state  objectives  in 
terms  of  learner  behavior,  assist  in 
developing  instruments  to  record 
behavior  by  reviewing  drafts  and 
providing  field  testing  oppor¬ 
tunities. 

Example:  A  Case  Study 

Case  study  analysis  can  be  used  to 
describe  participant  behavior  in 
educational  programs  where  social 
indicators  are  used  in  objectives. 


Prenatal  classes  provide  an  example. 
The  literature  on  becoming  a  parent 
for  the  first  time  indicates  that  first 
parenthood  is  a  crisis — a  time  of 
major  life  change.  New  parents  lack 
confidence  in  themselves  and  do 
not  understand  their  new  role. 

If  Extension  prenatal  classes  are 
designed  to  support  adult  learners 
by  helping  them  feel  capable  in 
assuming  parental  responsibilities 
and  able  to  understand  the  changes 
they  experience  during  expectancy 
and  early  parenthood,  then  they 
should  exhibit  self  confidence  and 
understanding  as  new  parents. 

Concepts  that  might  be  used  to 
describe  the  experience  of 
transition  to  first  parenthood  are 
drawn  from  ego  psychology  and 
from  learning  theory. 

Ego  psychology  differentiates 
coping  behavior  from  defensive 
behavior.  Theodore  Kroeber,  ego 
psychologist  and  author,  found  a 
way  to  measure  specific  kinds  of 
coping  and  defensive  behavior. 

Field  theory  describes  learning  as 
cognitive  change:  differentiation, 
generalization,  and  restructuriza¬ 
tion. 

Descriptions  Are  Important 

In  order  to  determine  program 
effect,  adults  can  be  interviewed  by 
using  questions  that  ask  them  to 
describe  feelings  about  themselves 
and  how  they  understand  the 
transition  they  are  experiencing. 
Clientele  descriptions  of  self  are 
valuable  in  describing  program 
outcome. 

Responses  to  questions  can  be 
analyzed  for  indicators  of  coping  or 
defensive  behavior.  If  the  amount  of 
coping  behavior  is  as  high  or  even 
higher  after  becoming  a  new  parent 
as  it  was  during  the  early  prenatal 
months,  participants  did  not  exhibit 
crisis  of  major  proportions  on  the 


Evaluation  Roles 

State  subject  matter  specialist  roles 
are  to  define  desired  learner 
behavior  in  measurable  terms;  draw 
on  conceptual  framework  to  explain 
the  desired  change  and  the  method 
of  contributing  to  change;  develop 
an  instrument  to  record  behavior  so 
that  the  changes  can  be  noted; 
analyze,  summarize,  and  interpret 
data;  and  disseminate  findings. 


examined  behaviors.  If  the  propor¬ 
tion  of  defensive  behavior  is  higher 
than  during  early  prenatal  months, 
participants  exhibited  an  inability  to 
adapt  to  their  new  roles. 

Field  theory  describes  learning  as 
cognitive  structure  changes. 
Responses  to  interview  questions 
regarding  understanding  the 
transition  can  be  analyzed  for 
indicators  of  differentiation,  gen¬ 
eralization,  and  restructurization. 
Differentiation  in  this  case  means 
clarifying  understanding  of  specific 
concepts.  Generalization  is  defined 
as  identifying  relationships  between 
concepts.  Restructurization  is 
defined  as  understanding  of  self 
using  the  newly  acquired 
knowledge. 

Use  of  Two  Bases 

Using  indicators  from  at  least  two 
different  theoretical  bases 
strengthens  the  validity  and 
reliability  of  explaining  participant 
behavior. 

Interviews  that  result  in  case  studies 
can  be  analyzed  for  indicators  of 
program  effect.  Responses  to  a  set 
of  interview  questions  can  be  used 
to  identify  program  participant 
behavior  or  social  consequences. 
Agents  and  specialists  design  the 
evaluation  methodology  together. 
Agents  conduct  interviews  to  collect 
data  and  state  specialists  analyze, 
summarize,  and  interpret  data.  The 
findings  provide  a  basis  for 
subsequent  programming.  □ 
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Obtaining  Data 

Midge  F.  Smith 

Extension  Program  Evaluation  Specialist 
University  of  Florida,  Gainesville 


Needs  assessment  is  a  process  which 
provides  the  basis  for  the  ultimate 
criterion  for  program  evaluation; 
did  the  program  lessen  the  need? 
Needs  assessments  are  also 
important  to  Extension  in  other 
ways: 

•  Program  development  involves 
designing  and  implementing  a 
course  of  action  to  bring  about 
change;  needs  assessment  is  a  tool 
for  deciding  rationally  what  that 
change  should  be; 

•  An  instructional  design  with 
measurable  behavioral  objectives 
provides  the  basis  for  determining  if 
programs  have  validity — do  they 
achieve  what  they  set  out  to 
achieve?  An  effective  needs  assess¬ 
ment  provides  the  basis  for  deter¬ 
mining  the  additional  criterion  of 
utility — was  the  achievement 
worthwhile? 

•  Needs  assessments  can  help  in 
Extension’s  accountability  and 
evaluation  thrust.  As  citizens 
identify  needs,  specify  required  out¬ 
comes,  and  agree  on  resources 
needed  to  achieve  the  outcomes, 
responsibility  can  begin  to  be 
shared  for  the  results.  As  all  partners 
agree  on  what  is  desired  and  work 
to  make  that  happen,  accountability 
becomes  a  shared  part  of  the  overall 
activity. 

Bases  for  Program  Priorities 

In  1973,  the  Extension  Committee 
on  Organization  and  Policy  (ECOP) 
identifed  the  following  four  bases 
for  educational  program  priorities: 

•  Expressed  needs  of  people. 

•  Analysis  of  environment  and 
other  conditions  of  society. 

•  Emerging  research  results. 


•  Administrative  response  to  recom¬ 
mendations  and  pressures  of 
support  groups. 

While  all  four  bases  are  important,  it 
is  the  first  two  which  must  have 
input  from  the  local  situation.  Exten¬ 
sion  agents  in  counties  can  depend 
on  state  and  federal  employees  to 
keep  them  informed  (at  least 
partially)  on  new  research  results 
and  on  administrative  responses. 
Their  critical  program  development 
challenge,  however,  is  to  identify 
the  problems,  issues,  situations,  or 
concerns  that  need  changing  or 
improving.  And  that  is  no  easy  task. 
Communities  constantly  change  as 
people  move  in  and  out,  through 
births  and  deaths,  and  as  individuals 
themselves  change  in  response  to 
social  and  economic  forces.  As  a 
result,  what  was  once  an 
appropriate  program  can  become 
inappropriate.  What  was  once  a 
defensible  program,  one  to  rally 
community  and  political  support, 
can  become  the  knell  which  rings 
decreased  program  support  and 
funding.  “Energy”  and  “environ¬ 
ment”  are  examples  of  programs 
that  have  moved  up  and  down  in 
priority  in  the  last  decade. 

Thus,  determining  public  needs  and 
making  value  judgments  in  selecting 
needs  as  a  basis  for  program 
development  may  be  the  most 
important  tasks  of  the  local  Exten¬ 
sion  agent.  They  may  also  be  the 
most  challenging  and  perplexing 
tasks. 

Needs  Assessment  Process 

In  Florida,  we  have  tried  to  decrease 
the  perplexity  by  providing  agents 
with  a  step-by-step  checklist  for 
identifying  and  prioritizing  citizen 
needs.  A  functional  description  of 
the  checklist  has  been  published  in 
a  manual. 

All  steps  are  illustrated  in  the 
manual  with  examples  from  a  com¬ 


prehensive  needs  assessment 
conducted  in  a  specific  county. 
Agents  are  encouraged  to  work 
through  all  the  appropriate  steps 
during  planning  so  that  the  full 
scope  of  the  work  and  needed  staff 
competencies  can  be  identified. 
Agents  check  off  the  steps  as  they 
occur  to  make  sure  key  procedures 
are  not  omitted. 

Three  methods  for  identifying 
information  about  citizen  needs  or 
problems  are  identified.  Each 
method  requires  increasingly  more 
resources  and  provides  more 
credible  data.  The  process  is 
cumulative,  too.  Number  1,  “Review 
Available  Information,”  should  be 
done  as  the  first  step  to  the  other 
methods;  and  number  2,  “Survey 
Key  Informants,”  should  be  done 
before  number  3,  “Survey  Target 
Population.”  The  final  step  in  each 
method  is  to  decide  if  adequate 
information  has  been  attained.  If 
the  answer  is  “YES”,  the  next  step  is 
to  begin  to  prioritize  the  needs.  If 
the  answer  is  “NO”,  the  next  step  is 
to  initiate  the  next  higher  level  of 
data  gathering  activity. 

The  level  at  which  the  agent  decides 
to  stop  in  gathering  data  will 
depend  on  available  resources  and 
the  purpose  of  the  needs 
assessment. 

The  next  part  of  the  checklist 
describes  procedures  an  agent  can 
use  with  advisory  committees,  key 
informants,  or  others  to  prioritize 
the  list  of  needs;  such  as  Delphi 
technique,  card  sorts,  scale  ratings, 
paired-weighting  procedure,  and 
budget  allocation  method. 

After  the  prioritization,  the  agent 
can  select  from  the  methods 
described  the  final  selection  of 
needs  for  development  of 
programs.  One  such  method — 
criteria  analysis — seeks  to  educate 
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the  user  on  important  considera¬ 
tions  in  selecting  needs  for  program 
development.  Eighteen  criteria  are 
described  which  affect  the  choices; 
for  example,  correlation  with 
Extension  mission  and  support  base; 
who  and  how  many  identified  the 
need;  expected  community 
support,  community  barriers,  and 
public  interest;  previous  success, 
expected  impact,  and  time  to  show 
results;  and  organizational  support. 

The  top  10  or  so  needs  are  analyzed 
against  each  criterion.  Current  pro¬ 
grams  (or  needs  they  address)  are 
also  analyzed  on  the  same  criteria 
and  at  the  same  time  to  put  new 
needs  in  perspective  with  current 
programing  efforts. 

The  final  selection  of  a  few  needs 
for  program  planning  completes  the 
needs  assessment  process  and 
begins  the  process  for  meaningful 
change  to  occur.  Unless  programs 
are  planned  and  implemented,  the 
needs  assessment  process  will  have 
been  a  wasted  effort.  It  may  have 
caused  harm  by  raising  the  expecta¬ 
tions  of  constituents.  Be  sure  there 
is  a  commitment  to  a  resulting 
program  before  starting  the  first 
step  in  identifying  needs. 

Potential  Errors 

Needs  assessments  provide  im¬ 
portant  data  for  local  program 
development  and  evaluation.  Every 
effort  should  be  made  to  secure 
data  as  free  of  bias  as  possible.  Some 
pitfalls  to  avoid  are  the  following: 

•  Imposing  our  own  orientations. 
This  is  the  problem  of  self-fulfilling 
prophecy;  that  is,  what  we  go 
looking  for,  we  find.  The  profes¬ 
sional  blinders  we  wear  affect  the 
questions  we  ask  and  what  we 
attend  to  in  the  answers.  The  key  is 
to  be  aware  of  our  limited  perspec¬ 
tives  and  to  make  every  effort  to  be 
objective. 


•  Paying  attention  only  to  surface 
problems.  These  problems  are  the 
easiest  to  identify  by  clients.  For 
more  lasting  change,  needs  assess¬ 
ments  should  provide  input  on  the 
underlying  causes  of  problems.  Final 
selection  of  problems  should  reflect 
Extension  agent  leadership  to  assure 
that  individuals  and  groups  involved 
are  well  informed  about  community 
trends,  new  technology,  and  so  on. 

•  Collecting  only  negative  informa¬ 
tion.  Focusing  only  on  weaknesses 
may  cause  clients  to  feel  hopeless. 
Strengths  also  need  to  be  identified 
as  a  foundation  on  which  to  base 
the  attack  on  problems. 

•  Conducting  the  assessment 
without  adequate  support.  Support 
involves  sanction  by  funding 
sources  of  Extension  programs, 
legitimizing  by  targeted  client 
groups,  and  direct  resources.  Many 
projects  are  not  completed  because 
unanticipated  demands  on  staff  time 
and  dollars  become  too  great. 


Unfortunately,  conclusions  from 
incomplete  studies  may  be 
inaccurate,  yet  harmful  because 
they  have  the  ring  of  credibility. 

It  is  important  for  Cooperative 
Extension  to  identify  and  select 
needs  for  program  development,  so 
that  Extension  can  help  people  solve 
their  problems  in  an  era  of  ever- 
shrinking  resources.  We  can  use 
needs  assessment  to  determine 
valid,  useful  programs  that  are 
philosophically  and  practically 
sound.  If  we  follow  a  rational 
process  in  which  people  and  the 
organization  reach  consensus  on 
how  resources  will  be  spent,  that  is, 
on  which  problems  will  be 
addressed,  we  ensure  that  our  time 
and  that  of  our  participants  is  spent 
in  mutually  valued  ways.  □ 
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RAP  Evaluates  Inservice  Education 

Bettie  Lee  Yerka 
Extension  Program  Coordinator 
Human  Ecology  Program 
Cornell  University,  Ithaca.  New  York 


Cornell  Cooperative  Extension 
recognizes  that  people  in  New  York 
State  who  make  or  influence  deci¬ 
sions  on  program  direction  and 
resources  need  evidence  on  results 
of  these  publicly  supported  educa¬ 
tional  efforts.  Staff  at  both  county 
and  state  levels  have  been 
encouraged  to  take  advantage, 
periodically  and  voluntarily,  of 
inservice  education  in  program 
evaluation,  as  well  as  graduate  study 
opportunities  outside  the  organiza¬ 
tion.  Although  national  surveys 
indicate  that  Extension  staff  most 
often  think  of  evaluation  for  pro¬ 
gram  improvement  purposes  and 
less  often  for  accountability,  the 
surveys  also  point  out  that  staff  may 
not  be  sufficiently  comfortable  with 
planning,  implementing,  and 
reporting  on  program  evaluations. 

Two  years  ago,  Cornell  Cooperative 
Extension  took  initial  steps  to  pro¬ 
vide  uniform  statewide  inservice 
education  in  program  evaluation. 
Teams  of  state  program  coordintors 
and  district  administrators,  having 
had  hands-on  experience  with  an 
evaluation  process  adaptable  to 
local  programing  situations,  have 
been  training  staff  across  the  state. 

The  process  selected.  Reflective 
Appraisal  of  Program  (RAP),  is  now 
demonstrating  its  potential  for  wide 
application  across  county  programs. 
Its  tested  standard  format  includes 
questions  focused  upon  a  model 
indicating  levels  of  Extension 
program  results  and  a  data 
collection  process  based  primarily 
on  telephone  interviewing.  In  New 
York  State,  the  RAP  format  adapts  to 
most  program  activities,  methods, 
and  clientele  within  Cornell  Co¬ 
operative  Extension. 

Revealing  Change 

Since  RAP  relies  upon  clientele's 
reflective  evidence  of  results  or 
changes  brought  about  by  a  given 
program,  it  can  also  reveal  par¬ 
ticipants’  estimates  of  the  extent  of 


that  change.  When  selected  pro¬ 
gram  activities  are  placed  into  the 
interview  questions.  Extension  staff 
can  fairly  efficiently  obtain  par¬ 
ticipants'  perceptions  of  the 
meaning  or  major  value  of  a 
program  to  them.  Levels  of  program 
results  are  particularly  significant  in 
this  evaluation  process. 

The  reflective  appraisal  approach  is 
useful  for  either  program  accounta¬ 
bility  or  improvement  purposes. 
Staff  are  involving  volunteers,  who 
are  leaders  and/or  program  com¬ 
mittee  members,  and  colleagues  at 
the  county,  district,  and  state  level 
in  planning  and  implementing  RAP 
studies.  Currently  RAP  is  also 
proving  helpful  in  determining  out¬ 
comes  of  some  similar  programs 
across  several  counties. 


•  Inner  city  tenants  concerned 
about  conditions  in  their  housing 
project 

•  4-H  participants  and  private 
sponsors  of  livestock  programs 

•  Farmers  examining  minimum 
tillage  techniques 

•  Homeowners  exploring  solar 
energy  practices 

•  4-H  members  and  leaders 
involved  in  community  service 
projects 

•  Participants  in  urban  home 
economics  programs 

•  Parents  of  youth  enrolled  in  a 
summer  nutrition  program 

•  Volunteers  in  food  preservation 

•  Participants  in  safe  wood  burning 
seminars 

•  Volunteer  Extension  program 
committee  members  in  several 
counties 

All  have  been  involved  in  evaluating 
Cooperative  Extension  programs  at 
the  county  level  during  the  past  2 
years.  They  may  have  been  ran¬ 
domly  selected  program  par¬ 


ticipants,  volunteer  interviewers,  or 
a  committee  of  volunteers  examin¬ 
ing  surveys  and  summa.  ies  of  pro¬ 
gram  results  and  identifying  positive 
factors  or  action  needed. 

Appraising  Results 

What  are  some  Extension  program 
results  documented  through 
reflective  appraisal?  A  few 
examples: 

•  Tenants  improving  shopping  and 
management  skills  and  relationships 
with  neighbors  and  landlords; 

•  4-H  members  identifying  skills 
they  will  use  on  the  family  farm,  in 
their  schools,  and  in  future 
marketing  activities; 

•  Farmers  becoming  more 
interested  in  minimum  tillage 
systems,  both  for  dollar  savings  and 
for  improving  cropping  practices; 

•  Energy  tour  participants  gaining 
more  knowledge  about  the  practical 
aspects  of  solar  heating  and  its 
potential  application  in  existing 
structures; 

•  4-H  members  learning  to  accept 
responsibility  for  themselves  and  for 
their  communities; 

•  Limited  income  families  changing 
the  quality  of  their  diets  through 
improved  planning,  purchasing,  and 
food  preparation  practices; 

•  Parents  of  young  children  apply¬ 
ing  skills  that  improve  communica¬ 
tions  with  their  children; 

•  Urban  families  learning  skills  in 
preserving  food  and  managing 
money; 

•  Parents  saying,  “Any  (class)  that 
increases  my  children's  desire  to 
consume  vegetables  is  enthusi¬ 
astically  accepted;’’  “My  child 
seems  more  conscious  about  what 
he  eats  now  and  how  much  he 
eats.’’ 

However,  evaluation  findings  have 
little  meaning  of  their  own.  Staff  in 
New  York  find  they  must  con¬ 
tinually  ask  questions  like  these: 
What  do  decision  makers  want  to 
know  about  our  programs?  What  do 


Who  is  involved? 

What  do  the  following  have  in 
common  in  New  York  State? 
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we  do?  Why  do  we?  Who  benefits? 

In  how  many  ways  can  we  use 
results  of  our  program  evaluation? 

Involving  people  in  the  evaluation 
process  is  a  major  dimension  of 
RAP.  Volunteers,  for  example, 
provide  an  articular  support  base. 
Volunteer  leaders  have  an  added 
investment;  programs  being 
evaluated  are  often  “theirs”  and  the 
results  are  important  to  them.  They 
are  helpful  in  collecting,  inter¬ 
preting  and  reporting  on  data.  For 
example,  volunteer,  program 
committee  members  have 
completed  100  interviews  in  one 
county  and  30  in  another;  volunteer 
program  committees  have  identified 
areas  for  improvement  and  possible 
solutions  for  each,  in — money 
management,  marketing,  and 
leadership  needs  in  4-H  livestock 
programs.  County,  district,  regional 
and  state  staff  contribute  to 
planning  and  carrying  out  of  RAP 
studies.  These  RAP  teams  provide  a 
means  for  program  staff  to  be 
involved  but  to  avoid  judgmental 
roles. 

Program  Selection 

Evaluation  needs  to  be  planned  in 
the  early  program  stages.  It  needs  to 
occur  at  a  time  when  clientele  have 
had  a  chance  to  modify  their 
behavior.  It  needs  to  fit  the  time 
constraints  of  staff.  RAP  is  not  a 
rigorous  research  procedure,  but  it 
does  require  effort.  Experience  with 
it  in  New  York  State  indicates  pro¬ 
grams  to  be  evaluated  need  to  be 
carefully  selected  and  levels  of 
evidence  identified  that  are  most 
needed  for  the  purpose  of  each 
evaluation.  And,  like  other 


evaluation  strategies,  RAP  should 
not  be  used  for  every  program 
every  year. 

RAP  provides  a  process  for  obtain¬ 
ing  clientele  perceptions  of  results 
important  to  them  that  occur  from  a 
variety  of  program  efforts.  However, 
one  process  does  not  encompass 
the  total  package  of  evaluation 
knowledge  and  skills  useful  for  all 
needs.  As  we  seek  appropriate  pro¬ 
gram  tools  for  the  80s,  we  need  to 
take  care  we  don’t  overdo — or 
underdo!  RAP  may  not  be  the  most 
appropriate  vehicle  to  generate 
documentation  of  Extension’s  worth 
to  people  unfamiliar  with  Extension, 
but  it  permits  staff  to  “see”  pro¬ 
grams  as  participants  do.  An  admin¬ 
istrator  in  another  state  describes  it 
as,  “a  simple,  defendable  approach 
to  systematically  gathering  informa¬ 
tion  for  decision  making.  .  .  .much 
better  information  than  what  is 
generally  used.” 

The  designer  of  RAP  considers  it  an 
evaluation  method  keyed  to  staff 
development;  it  extends  knowledge 
and  simplified  skills  to  those  in  the 
Extension  system  who  do  not  have 
opportunity  or  time  to  take  resident 
instruction  in  program  evaluation. 
County  level  accountability  is  an 
important  component  in 
Cooperative  Extension’s  public 
support  system  and  reflective 
evidence  is  generally  acceptable  to 
and  understood  by  staff,  volunteers, 
county  and  state  legislators,  and 
others  close  to  the  programing 
process. 

A  Stimulant  to  Evaluation 

RAP  assists  Cornell  Cooperative 
Extension  staff  at  state  and  county 


levels  to  perform  some  immediate 
roles  in  evaluation.  Its  concepts  and 
skills  are  easily  grasped:  program 
selection  and  description;  involve¬ 
ment;  levels  of  program  results; 
respondent  identification;  survey 
design;  data  collection  and  inter¬ 
pretation;  and  reporting.  Inservice 
education  in  RAP  has  stimulated 
evaluation  action.  Since  county  staff 
have  been  primarily  trained  to 
evaluate  their  highly  locally 
financed  programs,  state  staff 
provide  guidance  in  selected  multi¬ 
county  evaluation  efforts.  Some 
studies  will  produce  a  regional 
perspective,  and  will  strengthen 
documentation  of  program  results. 

Evaluation  and  accountability  are 
“writ  large”  in  Cooperative  Ex¬ 
tension’s  4-year  plan  of  work.  The 
recently  published  Extension  in  the 
1980s,  A  Report  of  the  joint  USDA- 
NASULCC  Committee  on  the  Future 
of  Cooperative  Extension  recom¬ 
mends  that  Cooperative  Extension 
involve  the  public  and  decision¬ 
makers  in  its  evaluation  efforts  in 
order  to  create  better  understand¬ 
ing.  Recommendations  also  suggest 
improved  reporting  to  the  public 
and  to  decisionmakers  at  all  levels. 

The  expectation  is  that  better 
informed  decisionmakers  will  make 
better  decisions  about  Cooperative 
Extension.  People  are  usually  more 
motivated  towards  program 
evaluation  processes  when  they  can 
see  their  relevance  to  program 
support  and  have  appropriate 
evaluation  tools.  RAP  can  make  a 
significant  contribution  to  program 
evaluation  in  Cooperative  Extension 
in  the  1980’s.  □ 
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